Image encoding / decoding method and method for transmitting bitstream

Abstract

An image encoding / decoding method and a method for transmitting a bitstream are provided. An image decoding method performed by an image decoding device according to the present disclosure comprises the steps of: determining a current block by dividing an image on the basis of division information obtained from a bitstream; determining a quantization parameter of the current block; and determining a transform coefficient of the current block based on the quantization parameter.

Description

[0001] This application is a divisional application of the invention patent application with the original application number 202080088773.3 (international application number: PCT / KR2020 / 014845, application date: October 28, 2020, invention name: Image encoding / decoding method and device using adaptive transformation and method for sending bit stream). Technical Field

[0002] The present disclosure relates to an image encoding / decoding method and apparatus, and more particularly to an image encoding / decoding method and apparatus using adaptive transform, and a method of transmitting a bit stream generated by the image encoding method / apparatus of the present disclosure. Background Art

[0003] Recently, the demand for high-resolution and high-quality images such as high-definition (HD) images and ultra-high-definition (UHD) images in various fields is increasing. As the resolution and quality of image data are improved, the amount of information or bit volume transmitted is relatively increased compared with existing image data. The increase in the amount of information or bit volume transmitted leads to an increase in transmission cost and storage cost.

[0004] Therefore, efficient image compression technology is needed to effectively transmit, store, and reproduce information about high-resolution and high-quality images. Summary of the invention

[0005] Technical issues

[0006] An object of the present disclosure is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0007] In addition, the present disclosure is directed to providing an image encoding / decoding method and apparatus for improving encoding / decoding efficiency by performing adaptive transformation.

[0008] In addition, the present disclosure is intended to provide a method of transmitting a bit stream generated by the image encoding method or apparatus according to the present disclosure.

[0009] In addition, the present disclosure is intended to provide a recording medium storing a bit stream generated by the image encoding method or apparatus according to the present disclosure.

[0010] In addition, the present disclosure is intended to provide a recording medium storing a bit stream received and decoded by the image decoding apparatus according to the present disclosure and used to reconstruct an image.

[0011] Those skilled in the art should understand that the technical objectives to be achieved by the present disclosure are not limited to the above-mentioned technical objectives, and other technical objectives not described herein will be clearly understood from the following description.

[0012] Technical Solution

[0013] According to one aspect of the present disclosure, there is provided an image decoding method performed by an image decoding device, the method comprising the following steps: determining a current block by dividing an image based on division information obtained from a bitstream; determining a quantization parameter of the current block; and determining a transform coefficient of the current block based on the quantization parameter. The determination of the quantization parameter may be performed based on whether a color space transform is applied to a residual sample of the current block, and the value of the quantization parameter may be determined to have a value within a range set from a predetermined upper limit to a predetermined lower limit.

[0014] In addition, according to an aspect of the present disclosure, there is provided an image decoding device, the image decoding device comprising a memory and at least one processor, wherein the at least one processor is configured to perform the following operations: determining a current block by dividing an image based on division information obtained from a bitstream; determining a quantization parameter of the current block; and determining a transform coefficient of the current block based on the quantization parameter, and the processor is configured to determine the quantization parameter based on whether a color space transform is applied to a residual sample of the current block. In this article, the value of the quantization parameter may be determined as a value within a range set from a predetermined upper limit to a predetermined lower limit.

[0015] In addition, according to an aspect of the present disclosure, there is provided an image encoding method performed by an image encoding device, the method comprising the following steps: determining a current block by dividing an image; determining a quantization parameter of the current block; and determining a transform coefficient of the current block based on the quantization parameter. In this article, the determination of the quantization parameter is determined based on whether a color space transform is applied to a residual sample of the current block, and the value of the quantization parameter can be determined to have a value within a range set from a predetermined upper limit to a predetermined lower limit.

[0016] In addition, according to another aspect of the present disclosure, a transmission method for transmitting a bit stream generated by the image encoding device or the image encoding method of the present disclosure is provided.

[0017] In addition, according to another aspect of the present disclosure, a computer-readable recording medium storing a bit stream generated by the image encoding method or the image encoding device of the present disclosure is provided.

[0018] The features described above in brief summary of the present disclosure are merely exemplary aspects of the following detailed description of the present disclosure and do not limit the scope of the present disclosure.

[0019] Beneficial Effects

[0020] According to the present disclosure, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency can be provided.

[0021] In addition, according to the present disclosure, it is possible to provide an image encoding / decoding method and apparatus capable of improving encoding / decoding efficiency by performing adaptive transformation.

[0022] In addition, according to the present disclosure, a method of transmitting a bit stream generated by the image encoding method or apparatus according to the present disclosure may be provided.

[0023] In addition, according to the present disclosure, a recording medium storing a bit stream generated by the image encoding method or apparatus according to the present disclosure can be provided.

[0024] In addition, according to the present disclosure, a recording medium storing a bit stream received and decoded by the image decoding device according to the present disclosure and used to reconstruct an image can be provided.

[0025] Those skilled in the art should understand that the effects that can be achieved by the present disclosure are not limited to the contents that have been specifically described above, and other advantages of the present disclosure will be more clearly understood from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Schematically shows a video encoding system to which an embodiment of the present disclosure is applicable.

[0027] Figure 2 Schematically illustrates an image encoding device to which an embodiment of the present disclosure is applicable.

[0028] Figure 3 Schematically illustrates an image decoding device to which an embodiment of the present disclosure is applicable.

[0029] Figure 4 is a diagram showing a segmentation structure of an image according to an embodiment.

[0030] Figure 5 is a diagram showing an embodiment of a partition type of a block according to a multi-type tree structure.

[0031] Figure 6 is a diagram illustrating a signaling mechanism of block partitioning information in a structure of a quadtree with nested multi-type trees according to the present disclosure.

[0032] Figure 7 A diagram showing an implementation of dividing a CTU into a plurality of CUs.

[0033] Figure 8 is a diagram illustrating neighboring reference samples according to an embodiment.

[0034] Fig. 9 and Fig.10 is a diagram illustrating intra prediction according to an embodiment.

[0035] Fig.11 is a diagram showing a block diagram of CABAC according to an embodiment for encoding one syntax element.

[0036] Figures 12 to 15 is a diagram illustrating entropy encoding and entropy decoding according to an embodiment.

[0037] Fig.16 and Fig.17 is a diagram showing an example of an image decoding and encoding process according to an embodiment.

[0038] Fig.18 is a diagram showing a hierarchical structure of a coded image according to an embodiment.

[0039] Fig.19 It is a diagram showing an embodiment of a decoding process to which ACT is applied.

[0040] Fig. 20 is a diagram illustrating an embodiment of a sequence parameter set syntax table for signaling syntax elements related to an ACT.

[0041] Figure 21 to Figure 27 It is a diagram showing an implementation of a syntax table for signaling a coding basis of syntax elements related to ACT in sequence.

[0042] Fig.28 is a diagram showing a coding tree syntax according to an embodiment.

[0043] Fig.29 is a diagram illustrating a method of encoding residual samples of BDPCM according to an embodiment.

[0044] Fig.30 is a diagram illustrating a modified quantized residual block generated by performing BDPCM according to an embodiment.

[0045] Fig.31 is a flowchart illustrating a process of encoding a current block by applying BDPCM in an image encoding apparatus according to an embodiment.

[0046] Fig.32 is a flowchart illustrating a process of reconstructing a current block by applying BDPCM in an image decoding apparatus according to an embodiment.

[0047] Figure 33 to Figure 35 is a diagram schematically showing syntax for signaling information on BDPCM.

[0048] Figure 36 to Figure 51 is a diagram showing a syntax table for signaling an ACT syntax element according to each individual embodiment of the present disclosure.

[0049] Fig.52 is a diagram illustrating an image decoding method according to an embodiment.

[0050] Fig.53 is a diagram illustrating an image encoding method according to an embodiment.

[0051] Fig.54 is a diagram showing a content streaming system to which an embodiment of the present disclosure is applicable. DETAILED DESCRIPTION

[0052] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to facilitate implementation by those skilled in the art. However, the present disclosure may be implemented in various forms and is not limited to the embodiments described herein.

[0053] When describing the present disclosure, if it is determined that the detailed description of related known functions or configurations makes the scope of the present disclosure unnecessarily ambiguous, the detailed description thereof will be omitted. In the drawings, parts irrelevant to the description of the present disclosure are omitted, and like reference numerals are given to like parts.

[0054] In the present disclosure, when a component is "connected", "coupled" or "linked" to another component, it may include not only a direct connection relationship but also an indirect connection relationship with intermediate components. In addition, when a component "includes" or "has" other components, unless otherwise specified, it means that other components may also be included, rather than excluding other components.

[0055] In the present disclosure, the terms first, second, etc. are used only for the purpose of distinguishing one component from other components, and do not limit the order or importance of the components unless otherwise specified. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment may be referred to as a first component in another embodiment.

[0056] In the present disclosure, components that are distinguished from each other are intended to clearly describe each feature and do not mean that the components must be separated. That is, multiple components can be integrated or implemented in one hardware or software unit, or one component can be distributed and implemented in multiple hardware or software units. Therefore, even if not specifically stated, implementations in which these components are integrated or components are distributed are also included in the scope of the present disclosure.

[0057] In the present disclosure, the components described in each embodiment are not necessarily indispensable components, and some components may be optional components. Therefore, the embodiments consisting of a subset of the components described in the embodiments are also included in the scope of the present disclosure. In addition, the embodiments including other components in addition to the components described in the various embodiments are included in the scope of the present disclosure.

[0058] The present disclosure relates to encoding and decoding of images. Unless otherwise defined in the present disclosure, terms used in the present disclosure may have general meanings commonly used in the technical field to which the present disclosure belongs.

[0059] In the present disclosure, "video" may refer to a group of images in a series according to the passage of time. "Picture" generally refers to the basis for representing an image within a specific time period, and a slice / tile is a coding basis that constitutes a part of a picture in coding. A picture may be composed of one or more slices / tiles. In addition, a slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / tiles. A picture may be composed of one or more tile groups. A tile group may include one or more tiles. A tile may refer to a quadrilateral area of ​​a CTU row within a tile in a picture. A tile may include one or more tiles. A tile may refer to a quadrilateral area of ​​a CTU row within a tile. A tile may be divided into multiple tiles, and each tile may include one or more CTU rows belonging to a tile. Tiles that are not divided into multiple tiles may also be regarded as tiles.

[0060] In the present disclosure, "pixel" or "picture element" may refer to the smallest unit constituting a picture (or image). In addition, "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a value of a pixel, and may represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component.

[0061] In the present disclosure, a "unit" may refer to a basic unit of image processing. The unit may include at least one of a specific area of ​​the picture and information related to the area. A unit may include a luminance block and two chrominance (e.g., Cb, Cr) blocks. In some cases, a unit may be used interchangeably with terms such as "sample array", "block" or "area". In general, an M×N block may include M columns and N rows of samples (or sample arrays) or a set (or array) of transform coefficients.

[0062] In the present disclosure, "current block" may refer to one of "current coding block", "current coding unit", "coding target block", "decoding target block" or "processing target block". When prediction is performed, "current block" may refer to "current prediction block" or "prediction target block". When transform (inverse transform) / quantization (dequantization) is performed, "current block" may refer to "current transform block" or "transform target block". When filtering is performed, "current block" may refer to "filtering target block".

[0063] In addition, in the present disclosure, unless explicitly stated as a chroma block, "current block" may refer to "luminance block of the current block." "Chroma block of the current block" may be expressed by including an explicit description of a chroma block such as "chroma block" or "current chroma block."

[0064] In the present disclosure, the terms " / " and "," should be interpreted as indicating "and / or". For example, the expressions "A / B" and "A, B" may mean "A and / or B". In addition, "A / B / C" and "A, B, C" may mean "at least one of A, B and / or C".

[0065] In the present disclosure, the term "or" should be interpreted to indicate "and / or". For example, the expression "A or B" may include 1) only "A", 2) only "B", and / or 3) both "A and B". In other words, in the present disclosure, the term "or" should be interpreted to indicate "additionally or alternatively".

[0066] Overview of Video Coding Systems

[0067] Figure 1 is a diagram illustrating a video encoding system according to the present disclosure.

[0068] The video encoding system according to the embodiment may include a source device 10 and a receiving device 20. The source device 10 may deliver encoded video and / or image information or data to the receiving device 20 via a digital storage medium or a network in the form of a file or a stream.

[0069] The source device 10 according to the embodiment may include a video source generator 11, an encoding device 12, and a transmitter 13. The receiving device 20 according to the embodiment may include a receiver 21, a decoding device 22, and a renderer 23. The encoding device 12 may be referred to as a video / image encoding device, and the decoding device 22 may be referred to as a video / image decoding device. The transmitter 13 may be included in the encoding device 12. The receiver 21 may be included in the decoding device 22. The renderer 23 may include a display and the display may be configured as a separate device or an external component.

[0070] The video source generator 11 can acquire video / images by capturing, synthesizing or generating a process of video / images. The video source generator 11 may include a video / image capturing device and / or a video / image generating device. The video / image capturing device may include, for example, one or more cameras, a video / image archive including previously captured videos / images, etc. The video / image generating device may include, for example, a computer, a tablet computer, and a smart phone, and may (electronically) generate a video / image. For example, a virtual video / image may be generated by a computer, etc. In this case, the video / image capturing process may be replaced by a process that generates relevant data.

[0071] The encoding device 12 may encode the input video / image. For compression and encoding efficiency, the encoding device 12 may perform a series of processes such as prediction, transformation, and quantization. The encoding device 12 may output encoded data (encoded video / image information) in the form of a bitstream.

[0072] The transmitter 13 may transmit the encoded video / image information or data output in the form of a bit stream to the receiver 21 of the receiving device 20 in the form of a file or stream through a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter 13 may include an element for generating a media file in a predetermined file format and may include an element for transmitting through a broadcast / communication network. The receiver 21 may extract / receive a bit stream from a storage medium or a network and transmit the bit stream to a decoding device 22.

[0073] The decoding device 22 may decode a video / image by performing a series of processes corresponding to the operations of the encoding device 12, such as dequantization, inverse transformation, and prediction.

[0074] The renderer 23 may render the decoded video / image. The rendered video / image may be displayed through a display.

[0075] Overview of Image Coding Equipment

[0076] Figure 2 Schematically illustrates an image encoding device to which an embodiment of the present disclosure is applicable.

[0077] like Figure 2 As shown, the image encoding device 100 may include an image segmenter 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-frame prediction unit 180, an intra-frame prediction unit 185, and an entropy encoder 190. The inter-frame prediction unit 180 and the intra-frame prediction unit 185 may be collectively referred to as a "prediction unit". The transformer 120, the quantizer 130, the dequantizer 140, and the inverse transformer 150 may be included in a residual processor. The residual processor may also include a subtractor 115.

[0078] In some embodiments, all or at least some of the components configuring the image encoding apparatus 100 may be configured by one hardware component (eg, an encoder or a processor). In addition, the memory 170 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium.

[0079] The image segmenter 110 may segment the input image (or picture or frame) input to the image encoding device 100 into one or more processing units. For example, the processing unit may be referred to as a coding unit (CU). The coding unit may be obtained by recursively segmenting a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree binary tree ternary tree (QT / BT / TT) structure. For example, a coding unit may be segmented into a plurality of coding units of a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. For the segmentation of the coding unit, the quadtree structure may be applied first, and then the binary tree structure and / or the ternary tree structure may be applied. The encoding process according to the present disclosure may be performed based on the final coding unit that is no longer segmented. The maximum coding unit may be used as the final coding unit, and the coding unit of a deeper depth obtained by segmenting the maximum coding unit may also be used as the final coding unit. Here, the encoding process may include the prediction, transformation, and reconstruction processes described later. As another example, the processing unit of the encoding process may be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transform unit may be divided or partitioned from the final coding unit. The prediction unit may be a sample prediction unit, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient.

[0080] The prediction unit (inter prediction unit 180 or intra prediction unit 185) may perform prediction on the block to be processed (current block) and generate a prediction block including prediction samples of the current block. The prediction unit may determine whether to apply intra prediction or inter prediction based on the current block or CU. The prediction unit may generate various information related to the prediction of the current block and transmit the generated information to the entropy encoder 190. The information about the prediction may be encoded in the entropy encoder 190 and output in the form of a bitstream.

[0081] The intra prediction unit 185 can predict the current block by referring to the samples in the current picture. Depending on the intra prediction mode and / or the intra prediction technology, the reference samples can be located in the neighbors of the current block or can be placed separately. The intra prediction mode may include multiple non-directional modes and multiple directional modes. The non-directional mode may include, for example, a DC mode and a plane mode. Depending on the level of detail of the prediction direction, the directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes. However, this is only an example, and more or fewer directional prediction modes may be used according to the settings. The intra prediction unit 185 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0082] The inter prediction unit 180 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc. The reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter prediction unit 180 may configure a motion information candidate list based on the neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter prediction unit 180 may use the motion information of the neighboring block as the motion information of the current block. In the case of skip mode, unlike the merge mode, the residual signal may not be transmitted. In the case of motion vector prediction (MVP) mode, the motion vector of the neighboring block may be used as a motion vector predictor, and the motion vector of the current block may be signaled by encoding the motion vector difference and an indicator of the motion vector predictor. The motion vector difference may mean the difference between the motion vector of the current block and the motion vector predictor.

[0083] The prediction unit may generate a prediction signal based on various prediction methods and prediction techniques described below. For example, the prediction unit may apply not only intra prediction or inter prediction, but also intra prediction and inter prediction simultaneously to predict the current block. The prediction method of applying both intra prediction and inter prediction simultaneously to predict the current block may be referred to as combined inter and intra prediction (CIIP). In addition, the prediction unit may perform intra block copying (IBC) to predict the current block. Intra block copying may be used for content image / video encoding of games, etc., such as screen content coding (SCC). IBC is a method of predicting the current picture using a previously reconstructed reference block in the current picture at a position separated by a predetermined distance from the current block. When IBC is applied, the position of the reference block in the current picture may be encoded as a vector (block vector) corresponding to a predetermined distance. IBC basically performs prediction in the current picture, but may be performed similarly to inter prediction because the reference block is derived within the current picture. That is, IBC may use at least one of the inter prediction techniques described in the present disclosure.

[0084] The prediction signal generated by the prediction unit can be used to generate a reconstruction signal or to generate a residual signal. The subtractor 115 can generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output from the prediction unit from the input image signal (original block or original sample array). The generated residual signal can be transmitted to the transformer 120.

[0085] The transformer 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a karhunen-loève transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, GBT refers to a transform obtained from a graph when relationship information between pixels is represented by a graph. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process may be applied to square pixel blocks of the same size or may be applied to blocks of variable size rather than square.

[0086] The quantizer 130 may quantize the transform coefficients and transmit them to the entropy encoder 190. The entropy encoder 190 may encode the quantized signal (information about the quantized transform coefficients) and output a bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantizer 130 may rearrange the quantized transform coefficients in the block form into a one-dimensional vector form based on the coefficient scanning order, and generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0087] The entropy encoder 190 may perform various encoding methods, such as exponential Golomb, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), etc. The entropy encoder 190 may encode information required for video / image reconstruction other than quantized transform coefficients together or separately (e.g., values ​​of syntax elements, etc.). The encoded information (e.g., encoded video / image information) may be transmitted or stored in units of a network abstraction layer (NAL) in the form of a bitstream. The video / image information may also include information about various parameter sets, such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. The signaled information, transmitted information, and / or syntax elements described in the present disclosure may be encoded and included in a bitstream through the above-described encoding process.

[0088] The bitstream may be transmitted over a network or may be stored in a digital storage medium. The network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) for transmitting a signal output from the entropy encoder 190 and / or a storage unit (not shown) for storing the signal may be included as an internal / external element of the image encoding device 100. Alternatively, a transmitter may be provided as a component of the entropy encoder 190.

[0089] The quantized transform coefficients output from the quantizer 130 may be used to generate a residual signal. For example, the residual signal (residual block or residual sample) may be reconstructed by applying dequantization and inverse transformation to the quantized transform coefficients through the dequantizer 140 and the inverse transformer 150.

[0090] The adder 155 adds the reconstructed residual signal to the prediction signal output from the inter-frame prediction unit 180 or the intra-frame prediction unit 185 to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). If the block to be processed has no residual, such as when the skip mode is applied, the prediction block can be used as a reconstructed block. The adder 155 can be called a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture, and can be used for inter-frame prediction of the next picture by filtering as described below.

[0091] The filter 160 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 160 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 170, specifically, in the DPB of the memory 170. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc. The filter 160 can generate various information related to filtering and transmit the generated information to the entropy encoder 190, as described later in the description of each filtering method. The information related to filtering can be encoded by the entropy encoder 190 and output in the form of a bit stream.

[0092] The modified reconstructed picture transferred to the memory 170 may be used as a reference picture in the inter prediction unit 180. When inter prediction is applied by the image encoding apparatus 100, prediction mismatch between the image encoding apparatus 100 and the image decoding apparatus may be avoided and encoding efficiency may be improved.

[0093] The DPB of the memory 170 may store the modified reconstructed picture for use as a reference picture in the inter-frame prediction unit 180. The memory 170 may store the motion information of the block from which the motion information in the current picture is derived (or encoded) and / or the motion information of the reconstructed block in the picture. The stored motion information may be transmitted to the inter-frame prediction unit 180 and used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 170 may store the reconstructed samples of the reconstructed blocks in the current picture and may transmit the reconstructed samples to the intra-frame prediction unit 185.

[0094] Overview of Image Decoding Equipment

[0095] Figure 3 Schematically illustrates an image decoding device to which an embodiment of the present disclosure is applicable.

[0096] like Figure 3 As shown, the image decoding device 200 may include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame prediction unit 260, and an intra-frame prediction unit 265. The inter-frame prediction unit 260 and the intra-frame prediction unit 265 may be collectively referred to as a "prediction unit". The dequantizer 220 and the inverse transformer 230 may be included in a residual processor.

[0097] According to an embodiment, all or at least some of the plurality of components configuring the image decoding apparatus 200 may be configured by hardware components (eg, a decoder or a processor). In addition, the memory 250 may include a decoded picture buffer (DPB) or may be configured by a digital storage medium.

[0098] The image decoding device 200 having received a bit stream including video / image information may perform the same Figure 2 The image encoding device 100 may reconstruct the image by processing corresponding to the processing performed by the image encoding device 100. For example, the image decoding device 200 may perform decoding using a processing unit applied in the image encoding device. Therefore, the processing unit of decoding may be, for example, a coding unit. The coding unit may be obtained by partitioning the coding tree unit or the maximum coding unit. The reconstructed image signal decoded and output by the image decoding device 200 may be reproduced by a reproduction device (not shown).

[0099] The image decoding device 200 may receive the image in the form of a bit stream from Figure 2The received signal may be decoded by the entropy decoder 210. For example, the entropy decoder 210 may parse the bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. The image decoding device may also decode the picture based on the information about the parameter set and / or the general constraint information. The information and / or syntax elements signaled / received described in the present disclosure may be decoded and obtained from the bitstream through a decoding process. For example, the entropy decoder 210 may decode the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients of the residual. More specifically, the CABAC entropy decoding method may receive a cell corresponding to each syntax element in a bitstream, determine a context model using information of a decoding target syntax element, information of a neighboring block and a decoding target block, or information of a symbol / cell decoded at a previous stage, perform arithmetic decoding on the cell by predicting the probability of occurrence of the cell according to the determined context model, and generate a symbol corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method may update the context model by using the information of the decoded symbol / cell for the context model of the next symbol / cell after determining the context model. Information related to prediction in the information decoded by the entropy decoder 210 may be provided to the prediction unit (inter-frame prediction unit 260 and intra-frame prediction unit 265), and the residual value on which entropy decoding is performed in the entropy decoder 210, that is, the quantized transform coefficient and related parameter information may be input to the dequantizer 220. In addition, information about filtering in the information decoded by the entropy decoder 210 may be provided to the filter 240. In addition, a receiver (not shown) for receiving a signal output from the image encoding apparatus may be further configured as an internal / external element of the image decoding apparatus 200 , or the receiver may be a component of the entropy decoder 210 .

[0100] In addition, the image decoding device according to the present disclosure may be referred to as a video / image / picture decoding device. The image decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 210. The sample decoder may include a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame prediction unit 260, or at least one of an intra-frame prediction unit 265.

[0101] The dequantizer 220 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 may rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement may be performed based on the coefficient scanning order performed in the image encoding device. The dequantizer 220 may dequantize the quantized transform coefficients by using a quantization parameter (e.g., quantization step size information) and obtain the transform coefficients.

[0102] The inverse transformer 230 may inversely transform the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0103] The prediction unit may perform prediction on the current block and generate a prediction block including prediction samples of the current block. The prediction unit may determine whether to apply intra prediction or inter prediction to the current block based on the information about prediction output from the entropy decoder 210, and may determine a specific intra / inter prediction mode (prediction technique).

[0104] The same as described in the prediction unit of the image encoding device 100 , the prediction unit can generate a prediction signal based on various prediction methods (techniques) described later.

[0105] The intra prediction unit 265 may predict the current block by referring to samples in the current picture. The description of the intra prediction unit 185 is also applicable to the intra prediction unit 265.

[0106] The inter-frame prediction unit 260 can derive the prediction block of the current block based on the reference block (reference sample array) specified by the motion vector on the reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter-frame prediction mode, the motion information can be predicted in units of blocks, sub-blocks or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter-frame prediction unit 260 may configure a motion information candidate list based on the neighboring blocks, and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes, and information about the prediction may include information indicating the inter-frame prediction mode of the current block.

[0107] The adder 235 can generate a reconstruction signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the prediction unit (including the inter-frame prediction unit 260 and / or the intra-prediction unit 265). If the block to be processed has no residual, for example, when the skip mode is applied, the prediction block can be used as the reconstructed block. The description of the adder 155 also applies to the adder 235. The adder 235 can be called a reconstructor or a reconstructed block generator. The generated reconstruction signal can be used for intra-frame prediction of the next block to be processed in the current picture, and can be used for inter-frame prediction of the next picture by filtering as described below.

[0108] The filter 240 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 240 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 250, specifically, in the DPB of the memory 250. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc.

[0109] The (modified) reconstructed picture stored in the DPB of the memory 250 may be used as a reference picture in the inter-frame prediction unit 260. The memory 250 may store the motion information of the block from which the motion information in the current picture is derived (or decoded) and / or the motion information of the reconstructed block in the picture. The stored motion information may be transmitted to the inter-frame prediction unit 260 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 250 may store the reconstructed samples of the reconstructed block in the current picture and transmit the reconstructed samples to the intra-frame prediction unit 265.

[0110] In the present disclosure, the implementations described in the filter 160, the inter-frame prediction unit 180 and the intra-frame prediction unit 185 of the image encoding device 100 can be equally or correspondingly applied to the filter 240, the inter-frame prediction unit 260 and the intra-frame prediction unit 265 of the image decoding device 200.

[0111] Overview of Image Segmentation

[0112] The video / image encoding method according to the present disclosure can be performed based on the image segmentation structure as follows. Specifically, the prediction, residual processing ((inverse) transform, (de)quantization, etc.), syntax element encoding and filtering processes described later can be performed based on the CTU, CU (and / or TU, PU) derived from the image segmentation structure. The image can be segmented in block units and the block segmentation process can be performed in the image segmentor 110 of the encoding device. The segmentation related information can be encoded by the entropy encoder 190 and sent to the image decoding device in the form of a bitstream. The entropy decoder 210 of the image decoding device can derive the block segmentation structure of the current picture based on the segmentation related information obtained from the bitstream, and based on this, a series of processes (e.g., prediction, residual processing, block / picture reconstruction, in-loop filtering, etc.) can be performed to perform image decoding.

[0113] A picture may be partitioned into a sequence of coding tree units (CTUs). Figure 4 An example of a picture being partitioned into CTUs is shown. A CTU may correspond to a coding tree block (CTB). Alternatively, a CTU may include a coding tree block of luma samples and two corresponding coding tree blocks of chroma samples. For example, for a picture containing three sample arrays, a CTU may include one N×N block of luma samples and two corresponding blocks of chroma samples.

[0114] Overview of CTU Segmentation

[0115] As described above, a coding unit may be obtained by recursively partitioning a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree / binary tree / ternary tree (QT / BT / TT) structure. For example, a CTU may be first partitioned into a quadtree structure. Thereafter, the leaf nodes of the quadtree structure may be further partitioned by a multi-type tree structure.

[0116] Partitioning according to the quadtree means that the current CU (or CTU) is equally divided into four. By partitioning according to the quadtree, the current CU can be partitioned into four CUs with the same width and the same height. When the current CU is no longer partitioned into a quadtree structure, the current CU corresponds to a leaf node of the quadtree structure. The CU corresponding to the leaf node of the quadtree structure may no longer be partitioned and may be used as the above-mentioned final coding unit. Alternatively, the CU corresponding to the leaf node of the quadtree structure may be further partitioned by a multi-type tree structure.

[0117] Figure 5 2 is a diagram illustrating partition types of a block according to a multi-type tree structure. The partitioning according to the multi-type tree structure may include two types of partitioning according to a binary tree structure and two types of partitioning according to a ternary tree structure.

[0118] The two types of splits according to the binary tree structure may include vertical binary split (SPLIT_BT_VER) and horizontal binary split (SPLIT_BT_HOR). Vertical binary split (SPLIT_BT_VER) means that the current CU is equally split into two in the vertical direction. Figure 4 As shown in FIG, through vertical binary splitting, two CUs with the same height as the current CU and half the width of the current CU can be generated. Horizontal binary splitting (SPLIT_BT_HOR) means that the current CU is equally divided into two in the horizontal direction. Figure 5 As shown, through horizontal binary partitioning, two CUs with a height half of the height of the current CU and the same width as the current CU can be generated.

[0119] The two types of splits according to the triad structure may include vertical triad split (SPLIT_TT_VER) and horizontal triad split (SPLIT_TT_HOR). In vertical triad split (SPLIT_TT_VER), the current CU is split in a vertical direction at a ratio of 1:2:1. Figure 5 As shown, through vertical trisection, two CUs with the same height as the current CU and a width of 1 / 4 of the current CU width and one CU with the same height as the current CU and a width of half the current CU width can be generated. In horizontal trisection (SPLIT_TT_HOR), the current CU is split in the horizontal direction at a ratio of 1:2:1. Figure 4 As shown, through horizontal trifurcated division, two CUs whose height is 1 / 4 of the height of the current CU and whose width is the same as the current CU and one CU whose height is half of the height of the current CU and whose width is the same as the current CU can be generated.

[0120] Figure 6 is a diagram illustrating a signaling mechanism of block partitioning information in a structure of a quadtree with nested multi-type trees according to the present disclosure.

[0121] Here, the CTU is regarded as the root node of the quadtree and is first split into a quadtree structure. Information (e.g., qt_split_flag) indicating whether quadtree partitioning is performed on the current CU (CTU or node (QT_node) of the quadtree) is notified by a signal. For example, when qt_split_flag has a first value (e.g., "1"), the current CU can be split by the quadtree. In addition, when qt_split_flag has a second value (e.g., "0"), the current CU is not split by the quadtree, but becomes a leaf node (QT_leaf_node) of the quadtree. Each quadtree leaf node can then be further split into a multi-type tree structure. That is, the leaf node of the quadtree can become a node (MTT_node) of a multi-type tree. In the multi-type tree structure, a first flag (e.g., Mtt_split_cu_flag) is notified by a signal to indicate whether the current node is additionally split. If the corresponding node is additionally split (for example, if the first flag is 1), the second flag (for example, Mtt_split_cu_vertical_flag) may be signaled to indicate the split direction. For example, the split direction may be a vertical direction when the second flag is 1, and a horizontal direction when the second flag is 0. Then, a third flag (for example, Mtt_split_cu_binary_flag) may be signaled to indicate whether the split type is a binary split type or a ternary split type. For example, the split type may be a binary split type when the third flag is 1, and a ternary split type when the third flag is 0. The nodes of the multi-type tree obtained by binary splitting or ternary splitting may be further split into a multi-type tree structure. However, the nodes of the multi-type tree may not be split into a quadtree structure. If the first flag is 0, the corresponding node of the multi-type tree is no longer split, but becomes a leaf node (MTT_leaf_node) of the multi-type tree. The CU corresponding to the leaf node of the multi-type tree may be used as the above-mentioned final coding unit.

[0122] Based on mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, a multi-type tree partitioning mode (MttSplitMode) of a CU may be derived as shown in the following Table 1. In the following description, a multi-tree partitioning mode may be referred to as a multi-tree partitioning type or simply a partitioning type.

[0123] [Table 1]

[0124] MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1

[0125] Figure 7 is a diagram showing an example of partitioning a CTU into a plurality of CUs by applying a multi-type tree after applying a quadtree. Figure 7 , the bold block edge 710 represents quadtree partitioning, and the remaining edge 720 represents multi-type tree partitioning. The CU may correspond to a coding block (CB). In an embodiment, the CU may include one coding block of luma samples and two coding blocks of chroma samples corresponding to the luma samples. The chroma component (sample) CB or TB size may be derived based on the luma component (sample) CB or TB size based on the component ratio according to the color format (chroma format, e.g., 4:4:4, 4:2:2, 4:2:0, etc.) of the picture / image. In the case of a 4:4:4 color format, the chroma component CB / TB size may be set equal to the luma component CB / TB size. In the case of a 4:2:2 color format, the width of the chroma component CB / TB may be set to half the width of the luma component CB / TB and the height of the chroma component CB / TB may be set to the height of the luma component CB / TB. In the case of a 4:2:0 color format, the width of the chroma component CB / TB may be set to half the width of the luma component CB / TB and the height of the chroma component CB / TB may be set to half the height of the luma component CB / TB.

[0126] In an embodiment, when the size of the CTU is 128 based on the luma sample unit, the size of the CU may have a size from 128×128 to 4×4, which is the same size as the CTU. In an embodiment, in the case of a 4:2:0 color format (or chroma format), the chroma CB size may have a size from 64×64 to 2×2.

[0127] Furthermore, in an embodiment, the CU size and the TU size may be the same. Alternatively, there may be multiple TUs in a CU region. The TU size generally indicates the luma component (sample) transform block (TB) size.

[0128] The TU size can be derived based on the maximum allowed TB size maxTbSize as a predetermined value. For example, when the CU size is larger than maxTbSize, multiple TUs (TBs) with maxTbSize can be derived from the CU, and transform / inverse transform can be performed in units of TU (TB). For example, the maximum allowed luma TB size may be 64×64 and the maximum allowed chroma TB size may be 32×32. If the width or height of the CB split according to the tree structure is larger than the maximum transform width or height, the CB may be automatically (or implicitly) split until the TB size limits in the horizontal and vertical directions are met.

[0129] In addition, for example, when intra prediction is applied, the intra prediction mode / type may be derived in units of CU (or CB), and the neighboring reference sample derivation and prediction sample generation process may be performed in units of TU (or TB). In this case, there may be one or more TUs (or TBs) in a CU (or CB) region, and in this case, multiple TUs or (TBs) may share the same intra prediction mode / type.

[0130] In addition, for a quadtree coding tree scheme with nested multi-type trees, the following parameters may be signaled from an image encoding device to a decoding device as SPS syntax elements. For example, at least one of a CTU size as a parameter representing a root node size of a quadtree, a MinQTSize as a parameter representing a minimum allowed quadtree leaf node size, a MaxBtSize as a parameter representing a maximum allowed binary tree root node size, a MaxTtSize as a parameter representing a maximum allowed ternary tree root node size, a MaxMttDepth as a parameter representing a maximum allowed hierarchical depth of multi-type tree partitioning from a quadtree leaf node, a MinBtSize as a parameter representing a minimum allowed binary tree leaf node size, or a MinTtSize as a parameter representing a minimum allowed ternary tree leaf node size may be signaled.

[0131] As an embodiment using a 4:2:0 chroma format, the CTU size may be set to 128×128 luminance blocks and two 64×64 chrominance blocks corresponding to these luminance blocks. In this case, MinOTSize may be set to 16×16, MaxBtSize may be set to 128×128, MaxTtSzie may be set to 64×64, MinBtSize and MinTtSize may be set to 4×4, and MaxMttDepth may be set to 4. Quadtree partitioning may be applied to the CTU to generate a quadtree leaf node. The quadtree leaf node may be referred to as a leaf QT node. The size of the quadtree leaf node may be from 16×16 size (e.g., MinOTSize) to 128×128 size (e.g., CTU size). If the leaf QT node is 128×128, it may not be additionally partitioned into a binary tree / ternary tree. This is because, in this case, even if it is partitioned, it exceeds MaxBtsize and MaxTtszie (e.g., 64×64). In other cases, the leaf QT node can be further split into a multi-type tree. Therefore, the leaf QT node is the root node of the multi-type tree, and the leaf QT node can have a multi-type tree depth (mttDepth) value of 0. If the multi-type tree depth reaches MaxMttdepth (for example, 4), further splitting can be ignored. If the width of the multi-type tree node is equal to MinBtSize and is less than or equal to 2xMinTtSize, further horizontal splitting can be ignored. If the height of the multi-type tree node is equal to MinBtSize and is less than or equal to 2xMinTtSize, further vertical splitting can be ignored. When splitting is not considered, the image encoding device can skip the signaling of the splitting information. In this case, the image decoding device can derive splitting information with a predetermined value.

[0132] In addition, one CTU may include a coding block of luma samples (hereinafter referred to as "luminance block") and two coding blocks of chroma samples corresponding thereto (hereinafter referred to as "chroma blocks"). The above coding tree scheme may be applied equally or individually to the luma block and chroma block of the current CU. Specifically, the luma block and chroma block in one CTU may be partitioned into the same block tree structure, and in this case, the tree structure is represented as SINGLE_TREE. Alternatively, the luma block and chroma block in one CTU may be partitioned into separate block tree structures, and in this case, the tree structure may be represented as DUAL_TREE. That is, when the CTU is partitioned into dual trees, the block tree structure for the luma block and the block tree structure for the chroma block may exist separately. In this case, the block tree structure for the luma block may be referred to as DUAL_TREE_LUMA, and the block tree structure for the chroma component may be referred to as DUAL_TREE_CHROMA. For P and B slices / block groups, the luma block and the chroma block in one CTU may be restricted to have the same coding tree structure. However, for I slices / patch groups, luma blocks and chroma blocks may have separate block tree structures from each other. If a separate block tree structure is applied, luma CTBs may be partitioned into CUs based on a specific coding tree structure, and chroma CTBs may be partitioned into chroma CUs based on another coding tree structure. That is, this means that a CU in an I slice / patch group to which a separate block tree structure is applied may include a coding block of a luma component or a coding block of two chroma components, and a CU of a P or B slice / patch group may include blocks of three color components (one luma component and two chroma components).

[0133] Although a quadtree coding tree structure with nested multi-type trees has been described, the structure for partitioning the CU is not limited thereto. For example, the BT structure and the TT structure may be interpreted as concepts included in a multi-partition tree (MPT) structure, and the CU may be interpreted as being partitioned by the QT structure and the MPT structure. In an example where the CU is partitioned by the QT structure and the MPT structure, a syntax element (e.g., MPT_split_type) including information about how many blocks a leaf node of the QT structure is partitioned into and a syntax element (e.g., MPT_split_mode) including information about which of the vertical and horizontal directions a leaf node of the QT structure is partitioned into may be signaled to determine the partition structure.

[0134] In another example, the CU may be partitioned in a manner different from the QT structure, the BT structure, or the TT structure. That is, unlike partitioning a CU of a lower depth into 1 / 4 of a CU of a higher depth according to the QT structure, partitioning a CU of a lower depth into 1 / 2 of a CU of a higher depth according to the BT structure, or partitioning a CU of a lower depth into 1 / 4 or 1 / 2 of a CU of a higher depth according to the TT structure, in some cases a CU of a lower depth may be partitioned into 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 of a CU of a higher depth, and the method of partitioning a CU is not limited thereto.

[0135] The quadtree coding block structure with multiple types of trees can provide a very flexible block segmentation structure. Due to the segmentation types supported in the multiple types of trees, different segmentation patterns can potentially produce the same coding block structure in some cases. In the image encoding device and decoding device, by limiting the occurrence of such redundant segmentation patterns, the amount of data of the segmentation information can be reduced.

[0136] In addition, in the encoding and decoding of the video / image according to the present document, the image processing basis may have a hierarchical structure. A picture may be divided into one or more tiles, tiles, slices and / or tile groups. A slice may include one or more tiles. A tile may include one or more CTU rows within a tile. A slice may include tiles of a picture, wherein the number of tiles is an integer. A tile group may include one or more tiles. A tile may include one or more CTUs. A CTU may be divided into one or more CUs. A tile may be a quadrilateral area consisting of a specific tile row and a specific tile column composed of multiple CTUs within a picture. A tile group may include tiles according to a tile grid scan within a picture, wherein the number of tiles is an integer. A slice header may carry information / parameters applicable to a corresponding slice (a block in a slice). When an encoding device or a decoding device has a multi-core processor, the encoding / decoding process for tiles, slices, tiles and / or tile groups may be performed in parallel.

[0137] In the present disclosure, the names or concepts of slices or patch groups may be used interchangeably. That is, a patch group header may be referred to as a slice header. In this document, a slice may have one of slice types including an intra (I) slice, a predicted (P) slice, and a bi-predicted (B) slice. For blocks within an I slice, inter prediction is not used for prediction, and only intra prediction may be used. Even in this case, the original sample values ​​may be encoded and signaled without prediction. For blocks within a P slice, intra prediction or inter prediction may be used. When inter prediction is used, only single prediction may be used. In addition, for blocks within a B slice, intra prediction or inter prediction may be used. When inter prediction is used, bidirectional prediction as the maximum range may be used.

[0138] According to the characteristics of the video image (e.g., resolution) or considering coding efficiency or parallel processing, the encoding device can determine the size of the patch / patch group, tile, slice, and the maximum and minimum coding unit. In addition, information about this or information for deriving this can be included in the bitstream.

[0139] The decoding device can obtain information indicating whether a tile / tile group, tile, slice, or CTU within a tile of the current picture is divided into multiple coding units. The encoding device and the decoding device only signal such information under certain conditions, thereby improving encoding efficiency.

[0140] The slice header (slice header syntax) may include information / parameters that are commonly applicable to the slice. APS (APS syntax) or PPS (PPS syntax) may include information / parameters that are commonly applicable to one or more pictures. SPS (SPS syntax) may include information / parameters that are commonly applicable to one or more sequences. VPS (VPS syntax) may include information / parameters that are commonly applicable to multiple layers. DPS (DPS syntax) may include information / parameters that are commonly applicable to the entire video. DPS may include information / parameters related to a combination of coded video sequences (CVS).

[0141] In addition, for example, information on the partitioning and configuration of tiles / tile groups / tiles / slices may be constructed via a high-level syntax during the encoding phase and sent to a decoding device in the form of a bitstream.

[0142] Overview of intra prediction

[0143] Hereinafter, intra prediction performed by the above encoding device and decoding device will be described in detail. Intra prediction may refer to prediction of generating a prediction sample of a current block based on a reference sample in a picture to which the current block belongs (hereinafter, current picture).

[0144] This will refer to Figure 8 When intra prediction is applied to the current block 801, neighboring reference samples to be used for intra prediction of the current block 801 may be derived. The neighboring reference samples of the current block may include: a total of 2×nH samples including a sample 811 adjacent to the left boundary of the current block of size nW×nH and a sample 812 adjacent to the lower left corner; a total of 2×nW samples including a sample 821 adjacent to the top boundary of the current block and a sample 822 adjacent to the upper right corner; and one sample 831 adjacent to the upper left corner of the current block. Alternatively, the neighboring reference samples of the current block may include multiple columns of top neighboring samples and multiple rows of left neighboring samples.

[0145] In addition, the neighboring reference samples of the current block may include: a total of nH samples 841 adjacent to the right boundary of the current block of size nW×nH; a total of nW samples 851 adjacent to the bottom boundary of the current block; and one sample 842 adjacent to the lower right corner of the current block.

[0146] However, some neighboring reference samples of the current block have not been decoded or may not be available. In this case, the decoding device can construct neighboring reference samples to be used for prediction by replacing unavailable samples with available samples. Alternatively, the neighboring reference samples to be used for prediction can be constructed by interpolation of available samples.

[0147] When deriving neighboring reference samples, (i) the prediction samples may be derived based on the average or interpolation of the neighboring reference samples of the current block, or (ii) the prediction samples may be derived based on reference samples existing in a specific (prediction) direction relative to the prediction samples among the neighboring reference samples of the current block. The case of (i) may be referred to as a non-directional mode or a non-angle mode, and the case of (ii) may be referred to as a directional mode or an angle mode. In addition, the prediction samples may be generated by interpolating the second neighboring samples and the first neighboring samples located in the opposite direction of the prediction direction of the intra-frame prediction mode of the current block based on the prediction samples of the current block among the neighboring reference samples. The above case may be referred to as linear interpolation intra-frame prediction (LIP). In addition, the chrominance prediction samples may be generated based on the luminance samples by using a linear model. This case may be referred to as an LM mode. In addition, the temporary prediction samples of the current block may be derived based on the filtered neighboring reference samples, and the prediction samples of the current block may be derived by weighted summing the temporary prediction samples and at least one reference sample derived according to the intra-frame prediction mode among the existing neighboring reference samples (i.e., the unfiltered neighboring reference samples). The above situation may be referred to as position-dependent intra prediction (PDPC). In addition, a reference sample row with the highest prediction accuracy may be selected from a plurality of neighboring reference sample rows of the current block to derive a prediction sample by using a reference sample of a prediction direction located in the corresponding row. At this time, intra prediction coding may be performed by indicating (notifying with a signal) the reference sample row used to the decoding device. The above situation may be referred to as multi-reference line (MRL) intra prediction or MRL-based intra prediction. In addition, the current block may be divided into vertical or horizontal sub-partitions to perform intra prediction based on the same intra prediction mode, and neighboring reference samples may be derived and used based on each sub-partition. That is, in this case, the intra prediction mode of the current block is equally applied to the sub-partitions, and neighboring reference samples are derived and used based on each sub-partition, thereby improving intra prediction performance in some cases. This prediction method may be referred to as intra sub-partitioning (ISP) or intra prediction based on ISP. These intra prediction methods may be referred to as intra prediction types distinguished from intra prediction modes (e.g., DC mode, plane mode, and directional mode). Intra prediction types may be referred to as various terms such as intra prediction schemes or additional intra prediction modes. For example, the intra prediction type (or additional intra prediction mode) may include at least one selected from the group of LIP, PDPC, MRL, and ISP described above. A general intra prediction method other than a specific intra prediction type such as LIP, PDPC, MRL, and ISP may be referred to as a normal intra prediction type. A normal intra prediction type may refer to a case where a specific intra prediction type is not applied, and prediction may be performed based on the above intra prediction mode. In addition, if necessary, the derived prediction sample may be post-filtered.

[0148] Specifically, the intra prediction process may include an intra prediction mode / type determination step, a neighboring reference sample derivation step, and a prediction sample derivation step based on the intra prediction mode / type. In addition, if necessary, a post-filtering step may be performed on the derived prediction samples.

[0149] In addition, in addition to the above-mentioned intra prediction types, affine linear weighted intra prediction (ALWIP) can also be used. ALWIP can be called linear weighted intra prediction (LWIP), or matrix weighted intra prediction or matrix-based intra prediction (MIP). When MIP is applied to the current block, the prediction sample of the current block can be derived by: i) using neighboring reference samples that have undergone an averaging process, ii) performing a matrix-vector multiplication process, and if necessary, also performing iii) a horizontal / vertical interpolation process. The intra prediction mode of MIP may be different from the intra prediction mode used in LIP, PDPC, MRL, ISP intra prediction, or normal intra prediction. The intra prediction mode of MIP may be called MIP intra prediction mode, MIP prediction mode, or MIP mode. For example, different matrices and offsets used in matrix-vector multiplication can be set according to the intra prediction mode of MIP. In this article, the matrix may be referred to as a (MIP) weighting matrix, and the offset may be referred to as a (MIP) offset vector or a (MIP) offset vector. The detailed MIP method will be described later.

[0150] The block reconstruction process based on intra prediction and intra prediction unit in the encoding device may schematically include, for example, the content described below. Step S910 may be performed by the intra prediction unit 185 of the encoding device. Step S920 may be performed by a residual processor, which includes at least one selected from the group of the subtractor 115, the transformer 120, the quantizer 130, the dequantizer 140 and the inverse transformer 150 of the encoding device. Specifically, step S920 may be performed by the subtractor 115 of the encoding device. At step S930, the prediction information may be derived by the intra prediction unit 185 and may be encoded by the entropy encoder 190. At step S930, the residual information may be derived by the residual processor and may be encoded by the entropy encoder 190. The residual information is information about the residual sample. The residual information may include information about the quantized transform coefficient of the residual sample. As described above, the residual sample may be derived as a transform coefficient by the transformer 120 of the encoding device, and the transform coefficient may be derived as a quantized transform coefficient by the quantizer 130. Information about the quantized transform coefficients may be encoded by the entropy encoder 190 through a residual encoding process.

[0151] The encoding device may perform intra prediction on the current block in step S910. The encoding device derives the intra prediction mode / type of the current block, derives the neighboring reference samples of the current block, and generates prediction samples in the current block based on the intra prediction mode / type and the neighboring reference samples. In this article, the determination of the intra prediction mode / type, the derivation of the neighboring reference samples, and the generation process of the prediction sample may be performed simultaneously, or any one of the processes may be performed before the other processes. For example, although not shown, the intra prediction unit 185 of the encoding device may include an intra prediction mode / type determination unit, a reference sample derivation unit, and a prediction sample derivation unit. The intra prediction mode / type determination unit may determine the intra prediction mode / type of the current block, the reference sample derivation unit may derive the neighboring reference samples of the current block, and the prediction sample derivation unit may derive the prediction samples of the current block. In addition, when performing the prediction sample filtering process to be described later, the intra prediction unit 185 may also include a prediction sample filter. The encoding device may determine the mode / type applied to the current block among multiple intra prediction modes / types. The encoding apparatus may compare RD costs of intra prediction modes / types and determine an optimal intra prediction mode / type for the current block.

[0152] In addition, the encoding device may perform a prediction sample filtering process. Prediction sample filtering may be referred to as post-filtering. Through the prediction sample filtering process, some or all of the prediction samples may be filtered. In some cases, the prediction sample filtering process may be omitted.

[0153] In step S920, the encoding device may generate residual samples of the current block based on the (filtered) prediction samples. The encoding device may compare the prediction samples in the original samples of the current block based on the phase, and may derive the residual samples.

[0154] In step S930, the encoding device may encode image information including information about intra prediction (prediction information) and residual information about residual samples. The prediction information may include intra prediction mode information and intra prediction type information. The encoding device may output the encoded image information in the form of a bit stream. The output bit stream may be transmitted to a decoding device via a storage medium or a network.

[0155] The residual information may include a residual encoding syntax to be described later. The encoding device may derive a quantized transform coefficient by transforming / quantizing the residual sample. The residual information may include information about the quantized transform coefficient.

[0156] In addition, as described above, the encoding device can generate a reconstructed picture (including reconstructed samples and reconstructed blocks). To this end, the encoding device can perform dequantization / inverse transformation on the quantized transform coefficients and derive (modified) residual samples. The reason for performing dequantization / inverse transformation after transformation / quantization of the residual samples is to derive the same residual samples as the residual samples derived by the decoding device as described above. The encoding device can generate a reconstructed block including the reconstructed samples of the current block based on the predicted samples and the (modified) residual samples. Based on the reconstructed block, a reconstructed picture of the current picture can be generated. As described above, the in-loop filtering process can also be applied to the reconstructed picture.

[0157] The video / image decoding process based on intra prediction and intra prediction units in the decoding device may schematically include, for example, the contents described below. The decoding device may perform operations corresponding to those performed by the encoding device.

[0158] Steps S1010 to S1030 may be performed by the intra prediction unit 265 of the decoding device. The prediction information in step S1010 and the residual information in step S1040 may be obtained from the bitstream by the entropy decoder 210 of the decoding device. A residual processor including a dequantizer 220 or an inverse transformer 230 or both of the decoding device may derive residual samples of the current block based on the residual information. Specifically, the dequantizer 220 of the residual processor may perform dequantization based on the quantized transform coefficient derived from the residual information, and may derive the transform coefficient. The inverse transformer 230 of the residual processor may perform an inverse transform on the transform coefficient and may derive the residual sample of the current block. Step S1050 may be performed by the adder 235 or the reconstructor of the decoding device.

[0159] Specifically, the decoding device may derive the intra prediction mode / type of the current block based on the received prediction information (intra prediction mode / type information) at step S1010. The decoding device may derive the neighboring reference samples of the current block at step S1020. At step S1030, the decoding device may generate prediction samples in the current block based on the intra prediction mode / type and the neighboring reference samples. In this case, the decoding device may perform a prediction sample filtering process. Prediction sample filtering may be referred to as post filtering. Through the prediction sample filtering process, some or all of the prediction samples may be filtered. In some cases, the prediction sample filtering process may be omitted.

[0160] The decoding device may generate a residual sample of the current block based on the received residual information. The decoding device may generate a reconstructed sample of the current block based on the predicted sample and the residual sample, and may derive a reconstructed block including the reconstructed sample at step S1040. Based on the reconstructed block, a reconstructed picture of the current picture may be generated. As described above, the in-loop filtering process may also be applied to the reconstructed picture.

[0161] Herein, although not shown, the intra prediction unit 265 of the decoding device may include an intra prediction mode / type determination unit, a reference sample derivation unit, and a prediction sample derivation unit. The intra prediction mode / type determination unit may determine the intra prediction mode / type of the current block based on the intra prediction mode / type information obtained from the entropy decoder 210. The reference sample derivation unit may derive the neighboring reference samples of the current block. The prediction sample derivation unit may derive the prediction sample of the current block. In addition, when performing the above-mentioned prediction sample filtering process, the intra prediction unit 265 may further include a prediction sample filter.

[0162] For example, the intra prediction mode information may include flag information (e.g., intra_luma_mpm_flag) indicating whether the most probable mode (MPM) is applied to the current block or the residual mode is applied. When MPM is applied to the current block, the prediction mode information may also include index information (e.g., intra_luma_mpm_idx) indicating one of the intra prediction mode candidates (MPM candidates). The intra prediction mode candidates (MPM candidates) may be constructed as an MPM candidate list or an MPM list. In addition, when MPM is not applied to the current block, the intra prediction mode information may also include residual mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra prediction modes other than the intra prediction mode candidates (MPM candidates). The decoding device may determine the intra prediction mode of the current block based on the intra prediction mode information. A separate MPM list may be constructed for the above-mentioned MIP.

[0163] In addition, the intra prediction type information may be implemented in various forms. For example, the intra prediction type information may include intra prediction type index information indicating one of the intra prediction types. As another example, the intra prediction type information may include reference sample row information (e.g., intra_luma_ref_idx) indicating whether MRL is applied to the current block and which reference sample row is used when MRL is applied to the current block, ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether ISP is applied to the current block, ISP type information (e.g., intra_subpartitions_split_flag) indicating the partition type of the sub-partition when ISP is applied, flag information indicating whether PDCP is applied, or flag information indicating whether LIP is applied. In addition, the intra prediction type information may include a MIP flag indicating whether MIP is applied to the current block.

[0164] The intra-frame prediction mode information and / or the intra-frame prediction type information may be encoded / decoded by the encoding method described in this document. For example, the intra-frame prediction mode information and / or the intra-frame prediction type information may be encoded / decoded by entropy coding (e.g., CABAC, CAVLC) based on a truncated (rice) binary code.

[0165] Quantization / Dequantization

[0166] As described above, the quantizer of the encoding device may derive quantized transform coefficients by applying quantization to the transform coefficients. The dequantizer of the encoding device or the dequantizer of the decoding device may derive transform coefficients by applying dequantization to the quantized transform coefficients.

[0167] In the encoding and decoding of video / still images, the quantization ratio can be changed, and the compression ratio can be adjusted using the changed quantization ratio. From the perspective of implementation, in consideration of complexity, a quantization parameter (QP) can be used instead of directly using the quantization ratio. For example, a quantization parameter having an integer value of 0 to 63 can be used, and each quantization parameter value can correspond to an actual quantization ratio. In addition, the quantization parameter QP of the luminance component (luminance sample) Y and the quantization parameter QP of the chroma component (chroma samples) C Can be set to be different from each other.

[0168] In the quantization process, the transform coefficient C can be input and divided by the quantization ratio Qstep, and the quantized transform coefficient C' can be obtained on this basis. In this case, considering the computational complexity, the quantization ratio can be multiplied by the scale to become an integer form, and the value corresponding to the scale value can be shifted. Based on the product of the quantization ratio and the scale value, the quantization scale can be derived. That is, the quantization scale can be derived according to the QP. The quantization scale can be applied to the transform coefficient C, and on this basis, the quantized transform coefficient C' can be derived.

[0169] The dequantization process is the inverse process of the quantization process. The quantized transform coefficient C' can be multiplied by the quantization ratio Qstep, on the basis of which the reconstructed transform coefficient C" can be obtained. In this case, the level scale can be derived from the quantization parameter. The level scale can be applied to the quantized transform coefficient C', and on this basis, the reconstructed transform coefficient C" can be derived. Due to losses in the transformation and / or quantization process, the reconstructed transform coefficient C" may be slightly different from the original transform coefficient C. Therefore, the encoding device can perform dequantization in the same manner as the decoding device.

[0170] In addition, an adaptive frequency weighted quantization technique that adjusts the quantization strength according to the frequency can be applied. The adaptive frequency weighted quantization technique is a method of applying a quantization strength that varies with frequency. In the adaptive frequency weighted quantization, the quantization strength that varies with frequency can be applied by using a predefined quantization scaling matrix. That is, the above-mentioned quantization / dequantization process can also be performed based on the quantization scaling matrix. For example, different quantization scaling matrices can be used according to the size of the current block and / or whether the prediction mode applied to the current block to generate the residual signal of the current block is inter-frame prediction or intra-frame prediction. The quantization scaling matrix can be referred to as a quantization matrix or a scaling matrix. The quantization scaling matrix can be predefined. In addition, for frequency adaptive scaling, frequency-specific quantization scaling information for the quantization scaling matrix can be constructed / encoded by the encoding device and can be notified to the decoding device with a signal. The frequency-specific quantization scaling information can be referred to as quantization scaling information. The frequency-specific quantization scaling information can include scaling list data (scaling_list_data). The (modified) quantization scaling matrix can be derived based on the scaling list data. In addition, the frequency-specific quantization scaling information can include a flag information indicating whether there is scaling list data. Alternatively, when the scaling list data is signaled at a high level (eg, SPS), information indicating whether the scaling list data is modified at a lower level (eg, PPS or tile group header, etc.) is also included.

[0171] Transformation / Inverse Transformation

[0172] As described above, the encoding device may derive a residual block (residual sample) based on a block (prediction sample) predicted by intra / inter / IBC prediction, and may derive a quantized transform coefficient by applying transform and quantization to the derived residual sample. By being included in the residual coding syntax, information about the quantized transform coefficient (residual information) may be encoded and output in the form of a bitstream. The decoding device may obtain information about the quantized transform coefficient (residual information) from the bitstream, and may derive the quantized transform coefficient by performing decoding. The decoding device may derive the residual sample by dequantization / inverse transform based on the quantized transform coefficient. As described above, quantization / dequantization or transform / inverse transform or both may be omitted. When the transform / inverse transform is omitted, the transform coefficient may be referred to as a coefficient or a residual coefficient, and may also be referred to as a transform coefficient for consistency of expression. Whether the transform / inverse transform is omitted may be signaled based on a transform skip flag (e.g., transform_skip_flag). The first value (e.g., 0) of transform_skip_flag may indicate whether the transform is omitted is determined by another syntax element. A second value (eg, 1) of transform_skip_flag may indicate transform omission (eg, skipping).

[0173] Transformation / inverse transformation may be performed based on one or more transform kernels. For example, a multiple transform selection (MTS) scheme for performing transformation / inverse transformation may be applied. In this case, some of the multiple transform kernel sets may be selected and applied to the current block. The transform kernel may be referred to as various terms such as a transform matrix, a transform type, etc. For example, a transform kernel set may refer to a combination of a transform kernel in a vertical direction (vertical transform kernel) and a transform kernel in a horizontal direction (horizontal transform kernel).

[0174] Transformation / inverse transformation can be performed on a per CU or TU basis. That is, transformation / inverse transformation can be applied to residual samples in a CU or residual samples in a TU. The CU size and the TU size can be the same, or multiple TUs can exist in a CU area. In addition, the CU size can generally refer to the luminance component (sample) CB size. The TU size can generally refer to the luminance component (sample) TB size. The chrominance component (sample) CB or TB size can be derived based on the luminance component (sample) CB or TB size according to the component ratio of the color format (chrominance format, for example, 4:4:4, 4:4:4, 4:2:2, 4:2:0, etc.). The TU size can be derived based on maxTbSize. For example, when the CU size is greater than maxTbSize, multiple TUs (TBs) with maxTbSize can be derived from the CU, and transformation / inverse transformation can be performed on the basis of each TU (TB). maxTbSize can be considered when determining whether to apply various intra-frame prediction types (for example, ISP). Information about maxTbSize can be predetermined. Alternatively, information about maxTbSize may be generated and encoded by the encoding device and signaled to the encoding device.

[0175] Entropy Coding

[0176] As mentioned above Figure 2 As described above, some or all of the video / image information may be entropy encoded by entropy encoder 190. Figure 3 Some or all of the described video / image information may be entropy decoded by the entropy decoder 310. In this case, the video / image information may be encoded / decoded on a per-syntax element basis. In this document, encoding / decoding of information may include encoding / decoding by the method described in this paragraph.

[0177] Fig.11A block diagram of CABAC for encoding a syntax element is shown. In the encoding process of CABAC, first, when the input signal is a syntax element rather than a binary value, the input signal is converted into a binary value by binarization. When the input signal is already a binary value, the input signal bypasses the binarization. In this article, each binary number 0 or 1 that constitutes a binary value can be referred to as an information element. For example, the binary string (information element string) after binarization is 110, and each of 1, 1, and 0 is called an information element. The information element of a syntax element can refer to the value of the syntax element.

[0178] The binarized cell can be input to a conventional coding engine or a bypass coding engine. The conventional coding engine can assign a context model that applies a probability value to the corresponding cell, and can encode the cell based on the assigned context model. After encoding each cell, the conventional coding engine can update the probability model of the cell. The cell encoded in this way can be called a context-coded cell. The bypass coding engine can omit the process for estimating the probability of the input cell and the process for updating the probability model applied to the cell after encoding. In the case of the bypass coding engine, the input cell is encoded by applying a uniform probability distribution (e.g., 50:50) instead of assigning a context, thereby improving the coding rate. The cell encoded in this way can be called a bypass cell. The context model can be assigned and updated for each cell to be context-coded (conventional coding), and the context model can be indicated based on ctxidx or ctxInc. ctxidx can be derived on the basis of ctxInc. Specifically, for example, the context index (ctxidx) indicating the context model of each conventionally coded octet can be derived as the sum of a context index increment (ctxInc) and a context index offset (ctxIdxOffset). In this document, ctxInc that varies with the octet can be derived. ctxIdxOffset can be represented by the lowest value of ctxIdx. The lowest value of ctxIdx can be referred to as the initial value (initValue) of ctxIdx. ctxIdxOffset is a value that is generally used to distinguish the context model from other syntax elements, and the context model of a syntax element can be distinguished / derived based on ctxinc.

[0179] During entropy coding, it is determined whether to perform coding through a regular coding engine or a bypass coding engine, and the coding path can be switched. Entropy decoding can perform the same process as entropy coding in reverse order.

[0180] For example, the above entropy coding can be Fig.12 and Fig.13 Execute as in . Fig.12 and Fig.13, the encoding device (entropy encoder) can perform an entropy encoding process on the image / video information. The image / video information may include segmentation related information, prediction related information (e.g., inter / intra prediction classification information, intra prediction mode information, and inter prediction mode information), residual information, and in-loop filtering related information, or may include various syntax elements related thereto. Entropy encoding may be performed based on each syntax element. Fig.12 Steps S1210 to S1220 can be performed by the above Fig.12 The encoding device is performed by the entropy encoder 190.

[0181] The encoding device may perform binarization on the target syntax element in step S1210. In this article, binarization may be performed based on various binarization methods such as truncated Rice binarization and fixed length binarization, and the binarization method of the target syntax element may be predefined. The binarization process may be performed by the binarization unit 191 in the entropy encoder 190.

[0182] The encoding device may perform entropy encoding on the target syntax element in step S1220. The encoding device may perform encoding based on conventional encoding (based on context) or based on bypass encoding on the cell string of the target syntax element based on, for example, context adaptive arithmetic coding (CABAC) or context adaptive variable length coding (CAVLC). The output may be included in a bitstream. The entropy encoding process may be performed by an entropy encoding processor 192 in the entropy encoder 190. As described above, the bitstream may be transmitted to a decoding device via a (digital) storage medium or a network.

[0183] Reference Fig.14 and Fig.15 , the decoding device (entropy decoder) can decode the encoded image / video information. The image / video information may include segmentation related information, prediction related information (e.g., inter / intra prediction classification information, intra prediction mode information, and inter prediction mode information), residual information, and in-loop filtering related information, or may include various syntax elements related thereto. Entropy encoding may be performed based on each syntax element. Steps S1410 to S1420 may be performed by the above Figure 3 The entropy decoder 210 of the decoding device is executed.

[0184] The decoding device may perform binarization on the target syntax element in step S1410. In this article, binarization may be performed based on various binarization methods such as truncated Rice binarization and fixed length binarization, and the binarization method of the target syntax element may be predefined. The decoding device may derive an available cell string (cell string candidate) for an available value of the target syntax element through the binarization process. The binarization process may be performed by the binarization unit 211 in the entropy decoder 210.

[0185] The decoding device may perform entropy decoding on the target syntax element in step S1420. When the decoding device sequentially decodes and parses each cell of the target syntax element from the input bits in the bitstream, and the derived cell string may be compared with the available cell string of the syntax element. When the derived cell string is the same as one of the available cell strings, the value corresponding to the cell string may be derived as the value of the syntax element. If not, the next bit in the bitstream is further parsed and the above process is performed again. Through this process, the start bit or end bit of specific information (specific syntax element) in the bitstream is not used, but variable length bits are used to signal the information. In this way, relatively few bits are assigned low values, thereby improving overall coding efficiency.

[0186] The decoding device may perform context-based or bypass-based decoding on each cell in the cell string from the bitstream based on an entropy coding scheme such as CABAC or CAVLC. The entropy decoding process may be performed by an entropy decoding processor 212 in the entropy decoder 210. The bitstream may include various types of information for image / video decoding as described above. As described above, the bitstream may be transmitted to the decoding device via a (digital) storage medium or a network.

[0187] In this document, a table including syntax elements (syntax table) may be used to represent signaling of information from an encoding device to a decoding device. The order of syntax elements in a table including syntax elements used in this document may refer to the order in which the syntax elements are parsed from a bitstream. The encoding device may construct and encode the syntax table so that the decoding device parses the syntax elements in the parsing order. The decoding device may parse and decode the syntax elements of the syntax table from the bitstream in the parsing order, thereby obtaining the values ​​of the syntax elements.

[0188] General image / video encoding process

[0189] In image / video coding, the pictures constituting the image / video can be encoded / decoded according to a series of decoding orders. The picture order corresponding to the output order of the decoded pictures can be set to be different from the decoding order, and based on this, forward prediction and backward prediction can be performed when performing inter-frame prediction.

[0190] Fig.16 An example of an exemplary picture decoding process to which embodiments of this document are applicable is shown. Fig.16 In step S1610, the above reference Figure 3The entropy decoder 210 of the decoding device described in the present invention may be performed by the entropy decoder 210 of the decoding device described in the present invention. Step S1620 may be performed by a prediction unit including an intra prediction unit 265 and an inter prediction unit 260. Step S1630 may be performed by a residual processor including a dequantizer 220 and an inverse transformer 230. Step S1640 may be performed by an adder 235. Step S1650 may be performed by a filter 240. Step S1610 may include the information decoding process described in this document. Step S1620 may include the inter / intra prediction process described in this document. Step S1630 may include the residual processing process described in this document. Step S1640 may include the block / picture reconstruction process described in this document. Step S1650 may include the in-loop filtering process described in this document.

[0191] Reference Fig.16 , the picture decoding process may schematically include, as shown above with reference to Figure 3 As described, the process of obtaining image / video information from the bit stream (by decoding) in step S1610, the picture reconstruction process in steps S1620 to S1640, and the in-loop filtering process of the reconstructed picture in step S1650. The picture reconstruction process can be performed based on the prediction samples and residual samples obtained by the inter-frame / intra-frame prediction in step S1620 and the residual processing (dequantization and inverse transformation of the quantized transform coefficients) in step S1630 described in this document. For the reconstructed picture generated by the picture reconstruction process, a modified reconstructed picture can be generated by the in-loop filtering process. The modified reconstructed picture can be output as a decoded picture and can be stored in a decoded picture buffer or a memory 250 of a decoding device to be used as a reference picture in an inter-frame prediction process later when the picture is decoded. In some cases, the in-loop filtering process can be omitted. In this case, the reconstructed picture can be output as a decoded picture and can be stored in a decoded picture buffer or a memory 250 of a decoding device to be used as a reference picture in an inter-frame prediction process later when the picture is decoded. As described above, the in-loop filtering process in step S1650 may include a deblocking filtering process, a sample adaptive offset (SAO) process, an adaptive loop filter (ALF) process, and / or a bilateral filtering process. Some or all of them may be omitted. In addition, one or some of the deblocking filtering process, the sample adaptive offset (SAO) process, the adaptive loop filter (ALF) process, and the bilateral filter process may be applied sequentially, or all of them may be applied sequentially. For example, the deblocking filtering process may be applied to the reconstructed picture, and then the SAO process may be performed. Alternatively, for example, the deblocking filtering process may be applied to the reconstructed picture, and then the ALF process may be performed. This may be performed in the same manner as in the encoding device.

[0192] Fig.17An example of an exemplary picture encoding process to which embodiments of this document are applicable is shown. Fig.17 In step S1710, the above reference Figure 2 The step S1710 may include the intra / intra prediction process described in this document. The step S1720 may include the residual processing process described in this document. The step S1730 may include the information encoding process described in this document.

[0193] Reference Fig.17 , the picture encoding process may schematically include, as shown above with reference to Figure 2 Described, for information (e.g., prediction information, residual information and segmentation information) for picture reconstruction is encoded and the information is output in the form of a bitstream, for a process for generating a reconstructed picture of the current picture, and for applying an (optional) process of in-loop filtering to the reconstructed picture. The encoding device can derive (modified) residual samples from the quantized transform coefficients through the dequantizer 140 and the inverse transformer 150, and can generate a reconstructed picture based on the prediction samples and (modified) residual samples output in step S1710. The generated reconstructed picture can be the same as the reconstructed picture generated by the above-mentioned decoding device. The in-loop filtering process can be performed on the reconstructed picture to generate a modified reconstructed picture. The modified reconstructed picture can be stored in the decoded picture buffer or memory 170. Similar to the case in the decoding device, the modified reconstructed picture can be used as a reference picture in the inter-frame prediction process later when the picture is encoded. As described above, in some cases, some or all of the in-loop filtering process can be omitted. When the in-loop filtering process is performed, (loop) filtering related information (parameters) can be encoded by the entropy encoder 190 and output in the form of a bitstream. The decoding device may perform an in-loop filtering process based on the filtering-related information in the same manner as the encoding device.

[0194] Through this in-loop filtering process, noise (e.g., block artifacts and ringing artifacts) generated during image / video encoding can be reduced, and subjective / objective visual quality can be improved. In addition, both the encoding device and the decoding device perform the in-loop filtering process, so that the encoding device and the decoding device can derive the same prediction result, the reliability of image encoding can be increased, and the amount of data transmitted for picture encoding can be reduced.

[0195] As described above, the picture reconstruction process can be performed in a decoding device as well as an encoding device. A reconstructed block can be generated based on intra-frame prediction / inter-frame prediction according to each block, and a reconstructed picture including the reconstructed block can be generated. When the current picture / slice / patchwork group is an I picture / slice / patchwork group, the blocks included in the current picture / slice / patchwork group can be reconstructed based only on intra-frame prediction. In addition, when the current picture / slice / patchwork group is a P or B picture / slice / patchwork group, the blocks included in the current picture / slice / patchwork group can be reconstructed based on intra-frame prediction or inter-frame prediction. In this case, inter-frame prediction can be applied to some blocks in the current picture / slice / patchwork group, and intra-frame prediction can be applied to some remaining blocks. The color components of the picture may include a luminance component and a chrominance component. Unless explicitly defined in this document, the methods and embodiments proposed in this document may be applied to luminance components and chrominance components.

[0196] Examples of coding hierarchies and structures

[0197] The encoded video / image according to this document can be processed according to, for example, the coding hierarchy and structure described later.

[0198] Fig.18 is a diagram showing the hierarchical structure of a coded image. The coded image can be divided into a video coding layer (VCL) that handles the decoding process of the image and itself, a subsystem for transmitting and storing coded information, and a network abstraction layer (NAL) that exists between the VCL and the subsystem and is responsible for network adaptation functions.

[0199] In the VCL, VCL data including compressed image data (slice data) can be generated. Alternatively, a parameter set including information such as a picture parameter set (PPS), a sequence parameter set (SPS), and a video parameter set (VPS) or a supplemental enhancement information (SEI) message additionally required for image decoding processing can be generated.

[0200] In NAL, header information (NAL unit header) is added to the raw byte sequence payload (RBSP) generated in the VCL, so that the NAL unit can be generated. In this article, RBSP refers to slice data, parameter sets, and SEI messages generated in the VCL. The NAL unit header may include NAL unit type information specified according to the RBSP data included in the NAL unit.

[0201] As shown in the figure, NAL units can be divided into VCL NAL units and non-VCL NAL units according to the RBSP generated in VCL. VCL NAL units may refer to NAL units that include information about images (slice data). Non-VCL NAL units may refer to NAL units that include information required for decoding images (parameter sets or SEI messages).

[0202] Using the header information added according to the data standard of the subsystem, the VCL NAL unit and the non-VCL NAL unit can be transmitted over the network. For example, the NAL unit can be converted into a data form of a predetermined standard such as H.266 / VVC file format, real-time transport protocol (RTP), transport stream (TS), and the resulting data can be transmitted over various networks.

[0203] As described above, regarding the NAL unit, the NAL unit type may be specified according to the RBSP data structure included in the NAL unit, and information on the NAL unit type may be stored in the NAL unit header and signaled.

[0204] For example, according to whether the NAL unit includes information about the image (slice data), it is roughly classified into VCL NAL unit type and non-VCL NAL unit type. VCL NAL unit type can be classified according to the characteristics and type of the picture included in the VCL NAL unit, and non-VCL NAL unit type can be classified according to the type of parameter set.

[0205] The following takes the NAL unit type specified according to the type of parameter set included in the non-VCL NAL unit type as an example.

[0206] -APS (Adaptation Parameter Set) NAL unit: A NAL unit that includes APS

[0207] -DPS (Decoding Parameter Set) NAL unit: A NAL unit that includes a DPS

[0208] -VPS (Video Parameter Set) NAL unit: A NAL unit that includes a VPS

[0209] - SPS (Sequence Parameter Set) NAL unit: a NAL unit that includes an SPS,

[0210] -PPS (Picture Parameter Set) NAL unit: A NAL unit that includes a PPS

[0211] The above-mentioned NAL unit type has syntax information for the NAL unit type, and the syntax information may be stored in the NAL unit header and signaled. For example, the syntax information may be nal_unit_type, and the NAL unit type may be specified by the nal_unit_type value.

[0212] A slice header (slice header syntax) may include information / parameters that are commonly applicable to slices. APS (APS syntax) or PPS (PPS syntax) may include information / parameters that are commonly applicable to one or more slices or pictures. SPS (SPS syntax) may include information / parameters that are commonly applicable to one or more sequences. VPS (VPS syntax) may include information / parameters that are commonly applicable to multiple layers. DPS (DPS syntax) may include information / parameters that are commonly applicable to the entire video. DPS may include information / parameters related to the concatenation of coded video sequences (CVS). In this document, a high-level syntax (HLS) may include at least one selected from the group of APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, and slice header syntax.

[0213] In this document, image / video information encoded by an encoding device and notified to a decoding device in the form of a bitstream may include information related to segmentation within a picture, intra / inter prediction information, residual information, and loop filtering information, as well as information included in a slice header, information included in an APS, information included in a PPS, information included in an SPS, and / or information included in a VPS.

[0214] Overview of Adaptive Color Transform (ACT)

[0215] Adaptive Color Transform (ACT) is a color space transformation (conversion) technique that is used to remove unnecessary overlap between color components and has been used in the HEVC Screen Content Extension. This can also be applied to VVC.

[0216] In HEVC Screen Content Extension (HEVC SCC extension), ACT has been used to adaptively convert prediction residuals from existing color spaces to YCgCo color space. One of the two color spaces can be optionally selected by signaling an ACT flag for each transform basis.

[0217] For example, a first value of the flag (eg, 1) may indicate that the residual of the transformed basis is encoded in the original color space. A second value of the flag (eg, 1) may indicate that the residual of the transformed basis is encoded in the YCgCo color space.

[0218] Fig.19 FIG. 4 is a diagram showing an embodiment of a decoding process to which ACT is applied. Fig.19 In an implementation manner, motion compensated prediction may correspond to inter-frame prediction in the present disclosure.

[0219] like Fig.19 As shown, a reconstructed picture (or a reconstructed block, a reconstructed sample array, a reconstructed sample, a reconstructed signal) can be generated based on the predicted output value and the residual output value. In this article, the residual output value can be an inverse transform output value. Here, the inverse transform can be a normal inverse transform. In this article, the normal inverse transform can be an inverse transform based on MTS or an inverse low frequency inseparable transform (LFNST).

[0220] In this context, the prediction output value may be a prediction block, a prediction sample array, a prediction sample, or a prediction signal. The residual output value may be a residual block, a residual sample array, a residual sample, or a residual signal.

[0221] For example, in the case of an encoding device, an ACT process may be performed on a residual sample derived based on a prediction sample. In addition, an output value of the ACT process may be provided as an input of a normal transform process. In this context, the normal transform process may be an MTS-based transform or a LFNST.

[0222] Information (parameters) about the (inverse) ACT may be generated and encoded by an encoding device, and may be transmitted to a decoding device in the form of a bitstream.

[0223] The decoding device may obtain, parse and decode (inverse) ACT related information (parameters), and may perform inverse ACT based on the (inverse) ACT related information (parameters).

[0224] On the basis of the inverse ACT, the residual sample (or residual block) can be derived (modified). For example, the (transform) coefficient can be derived by applying dequantization to the quantized (transform) coefficient. In addition, the residual sample can be derived by performing an inverse transform on the (transform) coefficient. In addition, the (modified) residual sample can be obtained by applying the inverse ACT to the residual sample. Information (parameters) about the (inverse) ACT will be described in detail later.

[0225] In an embodiment, the core transform function used in HEVC may be used as a core transform function (transform kernel) of color space transform. For example, matrices for forward transform and backward transform as shown in the following equations may be used.

[0226] [Formula 1]

[0227]

[0228] [Formula 2]

[0229]

[0230] Herein, C0, C1, and C2 may correspond to G, B, and R. Herein, G represents a green component, B represents a blue component, and R represents a red component. In addition, C0', C1', and C2' may correspond to Y, Cg, and Co. Herein, Y represents brightness, Cg represents green chrominance, and Co represents an orange chrominance component.

[0231] In addition, in order to compensate for the dynamic range change of the residual before and after the color transformation, QP adjustment can be applied to the transformed residual according to (-5, -5, -3). The details of QP adjustment will be described later.

[0232] Furthermore, in the encoding and decoding processes according to the embodiments, when ACT is applicable, the following restrictions may be applied.

[0233] - In case of dual-tree encoding / decoding, ACT is disabled. For example, ACT can be applied only to single-tree encoding / decoding.

[0234] - ACT can be disabled when ISP encoding and decoding is applied.

[0235] - ACT can be deactivated for chroma blocks with BDPCM applied. ACT can be activated only for luma blocks with BDPCM applied.

[0236] - CCLM can be deactivated when ACT can be applied.

[0237] Fig. 20 is a diagram illustrating an embodiment of a sequence parameter set syntax table for signaling syntax elements related to an ACT.

[0238] Figure 21 to Figure 27 It is a diagram showing an implementation of a syntax table for signaling a coding basis of syntax elements related to ACT in sequence.

[0239] like Fig. 20 As shown, as an ACT activation flag indicating whether ACT is activated in the decoding process, sps_act_enabled_flag 2010 can be used.

[0240] The first value (eg, 0) of sps_act_enabled_flag may indicate that ACT is not used, and that the flag cu_act_enabled_flag 2110 , 2710 indicating whether ACT is applied in the coding base is not provided in the syntax of the coding base.

[0241] A second value (eg, 1) of sps_act_enabled_flag may indicate that ACT may be used, and cu_act_enabled_flag may be provided in the syntax of the coding base.

[0242] When sps_act_enabled_flag is not obtained in a bitstream, the value of sps_act_enabled_flag may be derived as a first value (eg, 0).

[0243] In addition, if Fig.21 As shown, as an ACT flag indicating whether the residual of the current coding basis is encoded in the YCgCo color space, cu_act_enabled_flag 2110, 2710 can be used.

[0244] A first value of cu_act_enabled_flag (eg, 0) may indicate that the residual of the current coding basis is encoded in the original color space. A second value of cu_act_enabled_flag (eg, 1) may indicate that the residual of the current coding basis is encoded in the YCgCo color space.

[0245] When cu_act_enabled_flag is not provided in the bitstream, the flag may be derived as a first value (eg, 0). Herein, the original color space may be an RGB color space.

[0246] Transform-based QP derivation method using ACT QP offset

[0247] In an embodiment, in the scaling process of the transform coefficient, the derivation process of the quantization parameter and the Qp update process may be performed as follows: For example, the quantization parameter derivation process may be performed using the following parameters.

[0248] - Luma coordinates (xCb, yCb), which indicate the relative coordinates of the top left luma sample of the current coding block relative to the top left luma sample of the current picture,

[0249] - a variable cbWidth which indicates the width of the current coding block based on each luma sample,

[0250] - a variable cbHeight which indicates the height of the current coding block based on each luma sample,

[0251] - variable treeType, which indicates whether a single tree (SINGLE_TREE) or a dual tree is used to split the current coding tree node, and indicates whether the dual tree is a luma component dual tree (DAUL_TREE_LUMA) or a chroma component dual tree (DAUL_TREE_CHROMA) when a dual tree is used

[0252] In this process, the luma quantization parameter Qp'Y, and the chroma quantization parameters Qp'Cb, Qp'Cr, and Qp'CbCr may be derived.

[0253] The variable luma position (xQg, yQg) may indicate the position of the top left luma sample of the current quantization group corresponding to the top left sample of the current picture. In this paper, the horizontal position xQg and the vertical position yQg may be set equal to the values ​​of the variable CuQgTopLeftX and the variable CuQgTopLeftY, respectively. Fig.28 As shown, variables CuQgTopLeftX and CuQgTopLeftY may be defined as predetermined values ​​in the coding tree syntax.

[0254] In this paper, the current quantization group can be a quadrilateral area within the coding tree block and can share the same qP Y_PRED Its width and height may be equal to the width and height of the coding tree node in which the top left luma sample position is assigned to each of CuQgTopLeftX and CuQgTopLeftY.

[0255] When treeType is SINGLE_TREE or DUAL_TREE_LUMA, the luminance quantization parameter prediction value qP can be derived as follows: Y_PRED .

[0256] 1. The variable qP can be derived as follows Y_PRED .

[0257] (Condition 1) When any of the following conditions is true, qP Y_PRED The value is set to the same as SliceQp Y The same value (in this article, SliceQp Y Indicates the quantization parameter Qp of all slices in the picture Y Alternatively, qP Y_PRED The value of can be set to the luminance quantization parameter Qp of the last luminance coding basis of the immediately preceding quantization group according to the decoding order Y The value of .

[0258] -(Condition 1-1) When the current quantization group is the first quantization group in the slice

[0259] - (Condition 1-2) When the current quantization group is the first quantization group in the tile

[0260] - (Condition 1-3) When the current quantization group is the first quantization group in the CTB row in the tile and a predetermined synchronization occurs (eg, when the value of entropy_coding_sync_enabled_flag is 1)

[0261] 2. The variable qP can be derived as follows Y_A The value of .

[0262] -(Condition 2) When at least one of the following conditions is true, qP Y_A The value of qP is set Y_PRED Alternatively, qP Y_A The value of may be set to include the luma quantization parameter Qp of the coding basis for the luma coding block covering the luma sample position (xQg-1, yQg) Y The value of .

[0263] - (Condition 2-1) For the block identified by the sample position (xCb, yCb), the block identified by the sample position (xQg-1, yQg) is not an available neighboring block,

[0264] - (Condition 2-2) When the CTB of the luma coding block covering the luma sample position (xQg-1, yQg) is different from the CTB of the current luma coding block at the luma sample position (xCb, yCb), for example, when all of the following conditions are true

[0265] -(Condition 2-2-1)(xQg-1)>>CtbLog2SizeY value is different from (xCb)>>CtbLog2SizeY value

[0266] -(Condition 2-2-2) (yQg) >> CtbLog2SizeY value is different from (yCb) >> CtbLog2SizeY value

[0267] 3. The variable qP can be derived as follows Y_B The value of .

[0268] -(Condition 3) When at least one of the following conditions is true, qP Y_B The value of qP is set Y_PRED Alternatively, qP Y_B The value of may be set to include the luma quantization parameter Qp of the coding basis for the luma coding block covering the luma sample position (xQg, yQg-1). Y The value of .

[0269] - (Condition 3-1) For the block identified by the sample position (xCb, yCb), the block identified by the sample position (xQg, yQg-1) is not an available neighboring block,

[0270] - (Condition 3-2) When the CTB of the luma coding block covering the luma sample position (xQg, yQg-1) is different from the CTB of the current luma coding block at the luma sample position (xCb, yCb), for example, when all of the following conditions are true

[0271] -(Condition 3-2-1) The value of (xQg) >> CtbLog2SizeY is different from the value of (xCb) >> CtbLog2SizeY

[0272] -(Condition 3-2-2) (yQg-1) >> CtbLog2SizeY value is different from (yCb) >> CtbLog2SizeY value

[0273] 4. The brightness quantization parameter prediction value qP can be derived as follows Y_PRED .

[0274] When all of the following conditions are true, qP Y_PRED Set to the luma quantization parameter Qp of the coding basis for the luma coding block covering the luma sample position (xQg, yQg-1) Y .

[0275] - (Condition 3-1) For the block identified by the sample position (xCb, yCb), the block identified by the sample position (xQg, yQg-1) is an available neighboring block

[0276] - When the current quantization group is the first quantization group in the CTB row in the tile

[0277] In addition, when all conditions are not met, qP can be derived as follows Y_PRED .

[0278] [Formula 3]

[0279] Q Y_PRED =(qP Y_A +qP Y_B +1)>>1

[0280] Variable Qp Y It can be derived as shown below.

[0281] [Formula 4]

[0282] Q Y =((qP Y_PRED +CuQpDeltaVal+64+2*QpBdOffset)%(64+QpBdOffset))-QpBdOffset

[0283] In this article, CuQpDeltaVal indicates the difference between the luma quantization parameter used for coding basis and its predicted value. Its value can be obtained from the bitstream. QpBdOffset indicates the luma and chroma quantization parameter range offset. QpBdOffset can be preset to a predetermined constant or obtained from the bitstream. For example, QpBdOffset can be calculated by multiplying a predetermined constant by the value of a syntax element indicating the bit depth of luma or chroma samples. The luma quantization parameter Qp' can be derived as shown in the following formula Y .

[0284] [Formula 5]

[0285] Qp' Y =QpY+QpBdOffset

[0286] When the value of the variable ChromaArrayType indicating the type of the chroma array is not the first value (eg, 0) and treeType is SINGLE_TREE or DUAL_TREE_CHROMA, the following processing may be performed.

[0287] - When the value of treeType is DUAL_TREE_CHROMA, the value of the variable QpY can be set to the luma quantization parameter Qp corresponding to the luma coding basis covering the luma sample position (xCb+cbWidth / 2, yCb+cbHeight / 2). Y Same value.

[0288] -The variable qP can be derived as follows Cb ,qP Cr and qP CbCr .

[0289] [Formula 6]

[0290] Q Chroma =Clip3(-QpBdOffset, 63, Qp Y )

[0291] Q Cb =ChromaQpTable[0][qP Chroma ]

[0292] Q Cr =ChromaQpTable[1][qP Chroma ]

[0293] Q CbCr =ChromaQpTable[2][qP Chroma ]

[0294] The chromaticity quantization parameter Qp' of Cb and Cr components can be derived as follows: Cb and Qp' Cr And the chromaticity quantization parameter Qp' of the joint Cb-Cr coding CbCr .

[0295] [Formula 7]

[0296] Qp' Cb =Clip3(-QpBdOffset,63,qP Cb +pps_cb_qp_offset+slice_cb_qp_offset+CuQpOffset Cb )+QpBdOffset

[0297] Qp' Cr =Clip3(-QpBdOffset,63,qP Cr +pps_cr_qp_offset+slice_cr_qp_offset+CuQpOffset Cr )+QpBdOffset

[0298] Qp' CbCr =Clip3(-QpBdOffset,63,qP CbCr +pps_joint_cbcr_qp_offset+slice_joint_cbcr_qp_offset+CuQpOffset Cbcr )+QpBdOffset

[0299] In the above formula, pps_cb_qp_offset and pps_cr_qp_offset are used to derive Qp' Cb and Qp' Cr The offset can be obtained from the bitstream of the picture parameter set. slice_cb_qp_offset and slice_cr_qp_offset are used to derive Qp' Cb and Qp' Cr CuQpOffset is the offset of the slice header and can be obtained from the bitstream for the slice header. Cb and CuQpOffset Cr is used to derive Qp' Cb and Qp' Cr The offset can be obtained from the bitstream used for the transform basis.

[0300] In addition, for example, the dequantization process of the transform coefficient can be performed using the following parameters.

[0301] - Luma coordinates (xTbY, yTbY), which refers to the relative coordinates of the upper left corner sample of the current luma transform block relative to the upper left corner luma sample of the current picture

[0302] - variable nTbW, which indicates the transform block width

[0303] - variable nTbH, which indicates the transform block height

[0304] - The variable cIdx, which indicates the color component of the current block

[0305] The output of this process may be an array d of scaled transform coefficients. In this context, the size of array d may be (nTbW) x (nTbH). The individual elements that make it up may be identified as d[x][y].

[0306] To this end, the quantization parameter qP can be derived as follows: When the value of cIdx is 0, qP can be derived as shown in the following formula.

[0307] [Formula 8]

[0308] qP=Qp' Y

[0309] Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, it can be derived as shown in the following formula.

[0310] [Formula 9]

[0311] qP=Qp' CbCr

[0312] Alternatively, when the value of cIdx is 1, qP can be derived as shown in the following equation.

[0313] [Formula 10]

[0314] qP=Qp' Cb

[0315] Alternatively, when the value of cIdx is 2, qP can be derived as shown in the following equation.

[0316] [Formula 11]

[0317] qP=Qp' Cr

[0318] Afterwards, the quantization parameter qP may be updated as follows. In addition, the variables rectNonTsFlag and bdShift may be derived as follows. For example, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 0 (eg, when the transform of the current transform block is not skipped), the derivation may be performed as shown in the following equation.

[0319] [Formula 12]

[0320] qP=qP-(cu_act_enabled_fag[xTbY][yTbY]?5:0)

[0321] rectNonTsFlag=0

[0322] bdShift=10

[0323] Alternatively, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1 (eg, skipping the transformation of the current transform block), it may be derived as shown in the following equation.

[0324] [Formula 13]

[0325] qP=Max(QpPrimeTsMin, qP)-(cu_act_enabled_flag[xTbY][yTbY]?5:0)

[0326] rectNonTsFlag=(((Log2(nTbW)+Log2(nTbH))&1)==1

[0327] bdShift=BitDepth+(rectNonTsFlag?1:0)+((Log2(nTbW)+Log2(nTbH)) / 2)-+pic_dep_quant_enabled_flag

[0328] Herein, QpPrimeTsMin may indicate the minimum quantization parameter value allowed when the transform skip mode is applied. This may be determined as a predetermined constant or may be derived from a syntax element of the bitstream related thereto.

[0329] Herein, the suffixes Y, Cb, and Cr may denote G, B, and R color components in the RGB color model, or Y, Cg, and Co color components in the YCgCo color model.

[0330] Overview of Block Differential Pulse Code Modulation (BDPCM)

[0331] The image encoding device and the image decoding device according to the embodiment may perform differential encoding of the residual signal. For example, the image encoding device may encode the residual signal by subtracting the prediction signal from the residual signal of the current block, and the image decoding device may decode the residual signal by adding the prediction signal to the residual signal of the current block. The image encoding device and the image decoding device according to the embodiment may perform differential encoding of the residual signal by applying the following BDPCM.

[0332] The BDPCM according to the present disclosure may be performed in a quantized residual domain. The quantized residual domain may include a quantized residual signal (or a quantized residual coefficient), and when BDPCM is applied, the transformation of the quantized residual signal may be skipped. For example, when BDPCM is applied, the transformation of the residual signal may be skipped and quantization may be performed. Alternatively, the quantized residual domain may include a quantized transform coefficient.

[0333] In an embodiment of applying BDPCM, the image encoding device may derive a residual block of a current block predicted in an intra-frame prediction mode and quantize the residual block, thereby deriving the residual block. When a differential encoding mode of a residual signal is performed on the current block, the image encoding device may perform differential encoding on the residual block to derive a modified residual block. In addition, the image encoding device may encode differential encoding mode information specifying a differential encoding mode of a residual signal and a modified residual block, thereby generating a bitstream.

[0334] More specifically, when BDPCM is applied to the current block, a prediction block (predicted block) including a prediction sample of the current block can be generated by intra prediction. In this case, the intra prediction mode for performing intra prediction can be notified by a bit stream signal and can be derived based on the prediction direction of the BDPCM described below. In addition, in this case, the intra prediction mode can be determined as one of a vertical prediction direction mode or a horizontal prediction direction mode. For example, when the prediction direction of BDPCM is a horizontal direction, the intra prediction mode can be determined to be a horizontal prediction direction mode, and the prediction block of the current block can be generated by intra prediction in the horizontal direction. Alternatively, when the prediction direction of BDPCM is a vertical direction, the intra prediction mode can be determined to be a vertical prediction direction mode, and the prediction block of the current block can be generated by intra prediction in the vertical direction. When applying intra prediction in the horizontal direction, the value of the pixel adjacent to the left side of the current block can be determined as the predicted sample value of the sample included in the corresponding row of the current block. When intra prediction in the vertical direction is applied, the value of the pixel adjacent to the top of the current block can be determined as the predicted sample value of the sample included in the corresponding column of the current block. When BDPCM is applied to a current block, a method of generating a prediction block of the current block may be equally performed in an image encoding apparatus and an image decoding apparatus.

[0335] When BDPCM is applied to the current block, the image encoding device can generate a residual block including residual samples of the current block by subtracting the prediction sample from the current block. The image encoding device can quantize the residual block and then encode the difference (or increment) between the quantized residual sample and the prediction item of the quantized residual sample. The image decoding device can generate a quantized residual block of the current block by obtaining the quantized residual sample of the current block based on the prediction item and the difference reconstructed from the bitstream. Afterwards, the image decoding device can dequantize the quantized residual block and then add it to the prediction block to reconstruct the current block.

[0336] Fig.29 is a diagram illustrating a method of encoding residual samples of BDPCM according to the present disclosure. Fig.29 The residual block may be generated by subtracting the prediction block from the current block in an image encoding device. Fig.29 The quantized residual block can be generated by quantizing the residual block by the image encoding device. Fig.29 In, r i,j Specifies the value of the residual sample at the (i, j) coordinate in the current block. When the size of the current block is M×N, the value i can be from 0 to M-1, inclusive. In addition, the value j can be from 0 to N-1, inclusive. For example, the residual can refer to the difference between the original block and the predicted block. For example, r can be derived by subtracting the value of the predicted sample from the value of the original sample at the (i, j) coordinate in the current block. i,j For example, r i,j It can be the prediction residual after performing horizontal intra prediction or vertical intra prediction using samples that are not filtered from the top or left boundary samples. In horizontal intra prediction, the values ​​of the left neighboring pixels are copied along the lines that intersect the prediction block. In vertical intra prediction, the top neighboring rows are copied to separate rows of the prediction block.

[0337] exist Fig.29 In the i,j ) refers to the value of the quantized residual sample at coordinate (i, j) in the current block. For example, Q(r i,j ) can refer to r i,j Quantified value of .

[0338] right Fig.29 BDPCM prediction is performed on the quantized residual samples, and a modified quantized residual block R' including the modified quantized residual samples r' having a size of M×N may be generated.

[0339] When the prediction direction of BDPCM is horizontal, the value r' of the modified quantized residual sample at coordinate (i, j) in the current block i,j It can be calculated as shown below.

[0340] [Formula 14]

[0341]

[0342] As shown in Equation 14, when the prediction direction of BDPCM is horizontal, the value Q(r 0,j ) is assigned to the value r' at coordinate (0,j) as is 0,j . The values ​​of other coordinates (i,j) r' i,j It can be derived as the value Q(r) of the quantized residual sample at coordinate (i, j) i,j) and the value Q(r) of the quantized residual sample at coordinate (i-1, j) i-1,j ). That is, it is not the value Q(r) of the quantized residual sample at coordinate (i, j). i,j ) is encoded, but the value Q(r i-1,j ) is used as the difference calculated as the predicted value to derive the modified quantized residual sample value r' i,j , then for the value r' i,j to encode.

[0343] When the prediction direction of BDPCM is vertical, the value of the modified quantized residual sample (r' i,j ) can be calculated as shown below.

[0344] [Formula 15]

[0345]

[0346] As shown in Equation 15, when the prediction direction of BDPCM is vertical, the value Q(r i,0 ) is assigned to the value r' at coordinate (i,0) as is i,0 . The values ​​of other coordinates (i,j) r' i,j It can be derived as the value Q(r) of the quantized residual sample at coordinate (i, j) i,j ) and the value Q(r) of the quantized residual sample at coordinate (i, j-1) i,j-1 ). That is, it is not the value Q(r) of the quantized residual sample at coordinate (i, j). i,j ) is encoded, but the value Q(r i,j-1 ) is used as the difference calculated as the predicted value to derive the modified quantized residual sample value r' i,j , then for the value r' i,j to encode.

[0347] As described above, the process of modifying the current quantized residual sample value by using the nearby quantized residual sample values ​​as the prediction value may be referred to as BDPCM prediction.

[0348] Finally, the image encoding device may encode the modified quantized residual block including the modified quantized residual samples, and may transmit the resulting block to the image decoding device.Herein, as described above, the transformation of the modified quantized residual block is not performed.

[0349] Fig.30 is a diagram showing a modified quantized residual block generated by executing the BDPCM of the present disclosure.

[0350] exist Fig.30 , horizontal BDPCM shows a modified quantized residual block generated according to equation 14 when the prediction direction of BDPCM is the horizontal direction. In addition, vertical BDPCM shows a modified quantized residual block generated according to equation 15 when the prediction direction of BDPCM is the vertical direction.

[0351] Fig.31 is a flowchart showing steps of encoding a current block by applying BDPCM in an image encoding apparatus.

[0352] First, when the current block as the encoding target block is input in step S3110, prediction may be performed on the current block in step S3120 to generate a prediction block. The prediction block in step S3120 may be an intra-frame prediction block, and the intra-frame prediction mode may be determined as described above. Based on the prediction block generated in step S3120, a residual block of the current block may be generated in step S3130. For example, the image encoding device may generate a residual block (the value of the residual sample) by subtracting the prediction block (the value of the prediction sample) from the current block (the value of the original sample). For example, by performing step S3130, a residual block may be generated. Fig.29 For the residual block generated in step S3130, quantization may be performed in step S3140 to generate a quantized residual block, and BDPCM prediction may be performed on the quantized residual block in step S3150. The quantized residual block generated as a result of performing step S3140 may be Fig.29 As a result of the BDPCM prediction in step S3150, a quantized residual block can be generated according to the prediction direction. Fig.30 The modified quantized residual block. Since the BDPCM prediction in step S3150 has been referred to Figure 29 to Figure 30 The description is made, so the detailed description thereof will be omitted. Afterwards, the image encoding apparatus may encode the modified quantized residual block in step S3160 to generate a bit stream. In this article, the transformation of the modified quantized residual block may be skipped.

[0353] Reference in image coding device Figure 29 to Figure 31 The described BDPCM operations may in turn be performed by an image decoding device.

[0354] Fig.32 is a flowchart illustrating a process of reconstructing a current block by applying BDPCM in an image decoding apparatus.

[0355] The image decoding device may obtain information (image information) required to reconstruct the current block from the bitstream in step S3210. The information required to reconstruct the current block may include information about the prediction of the current block (prediction information) and information about the residual of the current block (residual information). The image decoding device may perform prediction on the current block based on the information about the current block, and may generate a predicted block in step S3220. The prediction of the current block may be intra-frame prediction, and its detailed description is the same as that described above with reference to Fig.31 Same as described. Fig.32 , it is shown that step S3220 of generating the prediction block of the current block is performed before steps S3230 to S3250 of generating the residual block of the current block. However, no limitation is imposed thereon. After generating the residual block of the current block, the prediction block of the current block may be generated. Alternatively, the residual block of the current block and the prediction block of the current block may be generated simultaneously.

[0356] The image decoding apparatus may generate a residual block of the current block by parsing the residual information of the current block from the bitstream in step S3230. The residual block generated in step S3230 may be Fig.30 The modified quantized residual block is shown.

[0357] The image decoding device can decode the Fig.30 The modified quantized residual block performs BDPCM prediction to generate Fig.29 The BDPCM prediction in step S3240 is used to Fig.30 The modified quantized residual block is generated Fig.29 The process of quantizing the residual block of the image encoding device, which corresponds to the inverse process of step S3150 performed by the image encoding device. For example, when the differential coding mode information (e.g., bdpcm_flag) obtained from the bitstream indicates a differential coding mode in which differential coding of the residual coefficient is performed when BDPCM is applied, the image decoding device performs differential coding on the residual block to derive the modified residual block. Using the residual coefficient to be modified and the predicted residual coefficient, the image decoding device can modify at least one residual coefficient to be modified among the residual coefficients in the residual block. The predicted residual coefficient can be determined based on the prediction direction indicated by the differential coding direction information (e.g., bdpcm_dir_flag) obtained from the bitstream. The differential coding direction information can indicate a vertical direction or a horizontal direction. The image decoding device can assign a value obtained by adding the residual coefficient to be modified and the predicted residual coefficient to the position of the residual coefficient to be modified. In this article, the predicted residual coefficient may be a coefficient immediately before and adjacent to the residual coefficient to be modified in terms of the order according to the prediction direction.

[0358] Hereinafter, the BDPCM prediction in step S3240 performed by the image decoding apparatus will be described in more detail below. The decoding apparatus may calculate the quantized residual sample Q (r i,j ). For example, when the prediction direction of BDPCM is the horizontal direction, the image decoding device may generate a quantized residual block from the modified quantized residual block by using Equation 16.

[0359] [Formula 16]

[0360]

[0361] As defined in Equation 16, the value Q(r) of the quantized residual sample at coordinate (i, j) can be calculated by adding the values ​​of the modified quantized residual samples starting from coordinate (0, j) to coordinate (i, j). i,j ).

[0362] Alternatively, using Equation 17 instead of Equation 16, the value Q(r) of the quantized residual sample at coordinate (i, j) can be calculated. i,j ).

[0363] [Formula 17]

[0364]

[0365] Equation 17 is the inverse of Equation 14. According to Equation 17, the value Q(r) of the quantized residual sample at the coordinate (0, j) is 0,j ) is derived as the value r' of the modified quantized residual sample at coordinate (0, j) 0,j . Q(r i,j ) is derived as the value r' of the modified quantized residual sample at coordinate (i, j) i,j and the value Q(r i-1,j ). That is, by taking the value Q(r i-1,j ) as the predicted value and the difference r' i,j Add and derive the quantized residual sample value Q(r i,j ).

[0366] When the prediction direction of the BDPCM is a vertical direction, the image decoding apparatus may generate a quantized residual block from the modified quantized residual block by using Equation 18.

[0367] [Formula 18]

[0368]

[0369] As defined in Equation 18, the value Q(r) of the quantized residual sample at coordinate (i, j) can be calculated by adding the values ​​of the modified quantized residual samples starting from coordinate (i, 0) to coordinate (i, j). i,j ).

[0370] Alternatively, using Equation 19 instead of Equation 18, the value Q(r) of the quantized residual sample at coordinate (i, j) can be calculated. i,j ).

[0371] [Formula 19]

[0372]

[0373] Equation 19 is the inverse of Equation 15. According to Equation 19, the value Q(r) of the quantized residual sample at coordinate (i, 0) is i,0 ) is derived as the value r' of the modified quantized residual sample at coordinate (i,0) i,0 . Q(r i,j ) is derived as the value r' of the modified quantized residual sample at coordinate (i, j) i,j and the value Q(r i,j-1 ). That is, by taking the value Q(r i,j-1 ) as the predicted value and the difference r' i,j Add and derive the quantized residual sample value Q(r i,j ).

[0374] When a quantized residual block composed of quantized residual samples is generated by executing step S3240 according to the above method, the image decoding device performs dequantization on the quantized residual block in step S3250 to generate a residual block of the current block. When BDPCM is applied, the transformation of the current block is skipped as described above. Therefore, the inverse transformation of the dequantized residual block can be skipped.

[0375] Afterwards, the image decoding device may reconstruct the current block in step S3260 based on the prediction block generated in step S3220 and the residual block generated in step S3250. For example, the image decoding device may reconstruct the current block (the value of the reconstructed sample) by adding the prediction block (the value of the prediction sample) and the residual block (the value of the residual sample). -1 (Q(r i,j)) is added to the intra-block prediction value to generate a reconstructed sample value. Differential coding mode information indicating whether BDPCM is applied to the current block can be notified by a signal through the bitstream. In addition, when BDPCM is applied to the current block, differential coding direction information indicating the prediction direction of BDPCM can be notified by a signal through the bitstream. When BDPCM is not applied to the current block, the differential coding direction information may not be notified by a signal.

[0376] Figure 33 to Figure 35 is a diagram schematically showing a syntax for signaling information about BDPCM.

[0377] Fig.33 is a diagram showing the syntax of a sequence parameter set according to an embodiment for signaling BDPCM information. In an embodiment, all SPS RBSPs included in at least one access unit (AU) having a value of 0 as a temporal ID (TemporalId) or provided by external means may be set to be used before being referenced in a decoding process. In addition, an SPS NAL unit including an SPS RBSP may be set to have the same nuh_layer_id as the nuh_layer_id of a PPS NAL unit that references the SPS NAL unit. In CVS, all SPS NAL units having a specific sps_seq_parameter_set_id value may be set to have the same content. Fig.33 In the seq_parameter_set_rbsp() syntax of , the above-mentioned sps_transform_skip_enable_flag and the later-described sps_bdpcm_enabled_flag are disclosed.

[0378] The syntax element sps_bdpcm_enabled_flag may indicate whether intra_bdpcm_flag is provided in the CU syntax for an intra coding unit. For example, a first value of sps_bdpcm_enabled_flag (e.g., 0) may indicate that intra_bdpcm_flag is not provided in the CU syntax for an intra coding unit. A second value of sps_bdpcm_enabled_flag (e.g., 1) may indicate that intra_bdpcm_flag is provided in the CU syntax for an intra coding unit. In addition, when sps_bdpcm_enabled_flag is not provided, the value of sps_bdpcm_enabled_flag may be set to a first value (e.g., 0).

[0379] Fig.34is a diagram showing an embodiment of a syntax for signaling whether BDPCM constraints are applied. In an embodiment, a general_constraint_info() syntax may be used to signal a predetermined constraint condition in the encoding / decoding process. Fig.34 The syntax of the sps_bdpcm_enabled_flag may signal a syntax element no_bdpcm_constraint_flag indicating whether the value of the above-mentioned sps_bdpcm_enabled_flag needs to be set to 0. For example, a first value (e.g., 0) of no_bdpcm_constraint_flag may indicate that such restriction is not applied. When the value of no_bdpcm_constraint_flag is a second value (e.g., 1), the value of sps_bdpcm_enabled_flag may be forced to be a first value (e.g., 0).

[0380] Fig.35 is a diagram illustrating an embodiment of a coding unit() syntax for signaling information about the BDPCM of a coding unit. Fig.35 As shown, the syntax elements intra_bdpcm_flag and intra_bdpcm_dir_flag may be signaled using the coding_unit() syntax. The syntax element intra_bdpcm_flag may indicate whether BDPCM is applied to the current luma coding block located at (x0, y0).

[0381] For example, a first value (e.g., 0) of intra_bdpcm_flag may indicate that BDPCM is not applied to the current luma coding block. A second value (e.g., 1) of intra_bdpcm_flag may indicate that BDPCM is applied to the current luma coding block. By indicating that BDPCM is applied, intra_bdpcm_flag may indicate whether to skip transform and whether to perform intra luma prediction mode by intra_bdpcm_dir_flag to be described later.

[0382] In addition, regarding x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1, the value of the above-mentioned variable BdpcmFlag[x][y] can be set to the value of intra_bdpcm_flag.

[0383] The syntax element intra_bdpcm_dir_flag may indicate the prediction direction of BDPCM. For example, a first value (eg, 0) of intra_bdpcm_dir_flag may indicate that the BDPCM prediction direction is a horizontal direction. A second value (eg, 1) of intra_bdpcm_dir_flag may indicate that the BDPCM prediction direction is a vertical direction.

[0384] Furthermore, with respect to x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1, the value of the variable BdpcmDir[x][y] may be set to the value of intra_bdpcm_dir_flag.

[0385] Intra prediction of chroma blocks

[0386] When intra prediction is performed on the current block, prediction of the luminance component block (luminance block) and prediction of the chrominance component block (chrominance block) of the current block can be performed. In this case, the intra prediction mode of the chrominance block can be set separately from the intra prediction mode of the luminance block.

[0387] For example, the intra prediction mode of the chroma block can be indicated based on the intra chroma prediction mode information. The intra chroma prediction mode information can be signaled in the form of a syntax element intra_chroma_pred_mode. For example, the intra chroma prediction mode information can indicate one of the following items: plane mode, DC mode, vertical mode, horizontal mode, derivation mode (DM) and cross component linear model (CCLM) mode. In this document, plane mode can refer to intra prediction mode 0, DC mode can refer to intra prediction mode 1, vertical mode can refer to intra prediction mode 26, and horizontal mode can refer to intra prediction mode 10. DM can also be called direct mode. CCLM can also be called linear model (LM). CCLM mode can include any of the following: L_CCLM, T_CCLM and LT_CCLM.

[0388] In addition, DM and CCLM are related intra prediction modes for predicting a chroma block by using information of a luma block. DM may refer to a mode in which the same intra prediction mode as the intra prediction mode of a luma component is applied as the intra prediction mode of a chroma component. In addition, CCLM may refer to an intra prediction mode in which, in a process of generating a prediction block of a chroma block, a reconstructed sample of a luma block is subsampled, and then, a sample derived by applying CCLM parameters α and β to the subsampled sample is used as a prediction sample of a chroma block.

[0389] Overview of the Cross-Component Linear Model (CCLM) model

[0390] As described above, the CCLM mode can be applied to the chrominance block. The CCLM mode is an intra-frame prediction mode that uses the correlation between the luminance block and the chrominance block corresponding to the luminance block, and is performed by deriving a linear model based on the neighboring samples of the luminance block and the neighboring samples of the chrominance block. In addition, the prediction sample of the chrominance block can be derived based on the derived linear model and the reconstructed sample of the luminance block.

[0391] Specifically, when the CCLM mode is applied to the current chrominance block, the parameters of the linear model can be derived based on the neighboring samples used for intra-frame prediction of the current chrominance block and the neighboring samples used for intra-frame prediction of the current luminance block. For example, the linear model of CCLM can be expressed based on the following formula.

[0392] [Formula 20]

[0393] pred c (i, j) = α·rec L ′(i, j)+β

[0394] In this paper, pred c (i, j) may refer to the predicted sample of the coordinate (i, j) of the current chroma block in the current CU. L '(i,j) may refer to the reconstructed sample of the coordinate (i,j) of the current luminance block in the CU. For example, rec L '(i,j) may refer to the downsampled reconstructed sample of the current luma block. The linear model coefficients α and β may be signaled or derived from neighboring samples.

[0395] Joint Coding of Residual (Joint CbCr)

[0396] In the encoding / decoding process according to the embodiment, the chroma residual can be encoded / decoded together. This can be called joint coding of the residual or joint CbCr. Whether the joint coding mode of CbCr is applied (activated) can be signaled by the joint coding mode signaling flag tu_joint_cbcr_residual_flag, which is signaled at the level of the transform basis. In addition, the selected coding mode can be derived by the chroma CBF. When the value of at least one chroma CBF used for the transform basis is 1, the flag tu_joint_cbcr_residual_flag may be present. The chroma QP offset value indicates the difference between the universal chroma QP offset value signaled for the conventional chroma residual coding mode and the chroma QP offset value for the CbCr joint coding mode. The chroma QP offset value can be signaled by the PPS or slice header. The QP offset value can be used to derive the chroma QP value of the block using the joint chroma residual coding mode.

[0397] In the case where mode 2 as the corresponding joint chroma coding mode in the following table is activated for transform basis, its chroma QP offset may be added to the target luma-derived chroma QP (applied luma-derived chroma QP) while performing quantization and decoding of the transform basis.

[0398] For other modes such as Mode 1 and Mode 3 in the table below, the chroma QP can be derived in the same way as the chroma QP is obtained for a common Cb or Cr block. This process of reconstructing the chroma residuals (resCb and resCr) from the transform blocks can be selected according to the table below. When the current mode is activated, a single joint chroma residual block (resJointC[x][y] in the table below) is signaled, and the residual block resCb of Cb and the residual block resCr of Cr can be derived taking into account information (e.g., tu_cbf_cb, tu_cbf_cr, and CSign as a symbol value disclosed in the slice header).

[0399] In the encoding device, the joint chrominance component can be derived as follows. According to the joint encoding method, resJointC{1,2} can be generated according to the following sequence. In the case of mode 2 (single residual with reconstruction Cb=C, Cr=CSign*C), the joint residual can be determined according to the following formula.

[0400] [Formula 21]

[0401] resJointC[x][y]=(resCb[x][y]+CSign*resCr[x][y]) / 2

[0402] Alternatively, in the case of Mode 1 (single residual with reconstruction Cb=C, Cr=(CSign*C) / 2), the joint residual may be determined according to the following equation.

[0403] [Formula 22]

[0404] resJointC[x][y]=(4*resCb[x][y]+2*CSign*resCr[x][y]) / 5

[0405] Alternatively, in the case of mode 3 (single residual with reconstruction Cr=C, Cb=(CSign*C) / 2), the joint residual can be determined according to the following equation.

[0406] [Formula 23]

[0407] resJointC[x][y]=(4*resCr[x][y]+2*CSign*resCb[x][y]) / 5

[0408] [Table 2]

[0409]

[0410] The above table shows the reconstruction of the chroma residual. CSign refers to the symbol value +1 or -1 specified in the slice header. resJointC[][] refers to the transmitted residual. In the table, the mode refers to TuCResMode described later. Only for I slices, the three joint chroma coding modes in the table may be supported. For P and B slices, only mode 2 may be supported. Therefore, for P and B slices, the syntax element tu_joint_cbcr_residual_flag can only be provided when the chroma cbf (e.g., tu_cbf_cb and tu_cbf_cr) values ​​are both 1. In addition, in the context modeling of tu_cbf_luma and tu_cbf_cb, the transform depth can be removed.

[0411] Implementation 1: QP update method using ACT Qp offset

[0412] As described above, an update of the QP for applying ACT may be performed. There are several problems with the above QP update. For example, when using the above method, it is not possible to set different ACT Qp offsets for individual color components. In addition, the derived qP value may have a negative value. Therefore, in the following embodiment, a method for applying clipping to the Qp value derived from the ACTQP offset value based on the color component value is described.

[0413] In an embodiment, the quantization parameter qP may be derived as follows.

[0414] First, when the value of cIdx is 0, qP and ACT Qp offset can be derived as shown in the following equation.

[0415] [Formula 24]

[0416] qP=Qp' Y

[0417] ActQpOffset=5

[0418] Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, the qP and ACTQp offsets may be derived as shown below.

[0419] [Formula 25]

[0420] qP=Qp' CbCr

[0421] ActQpOffset=5

[0422] Alternatively, when the value of cIdx is 1, qP and ACT Qp offset may be derived as shown below.

[0423] [Equation 26]

[0424] qP=Qp' Cb

[0425] ActQpOffset=3

[0426] The quantization parameter qP may be updated as follows.

[0427] When the value of transform_skip_flag[xTbY][yTbY][cIdx] is 0, qP can be derived as shown in the following equation.

[0428] [Formula 27]

[0429] qP=Max(0, qP-(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0))

[0430] Alternatively, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1, qP may be derived as shown in the following equation.

[0431] [Equation 28]

[0432] qP=Max(0,Max(QpPrimeTsMin,qP)-(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0))

[0433] In another embodiment, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1, qP may be clipped using the value of QpPrimeTsMin instead of 0, as shown in the following equation.

[0434] [Equation 29]

[0435] qP=Max(QpPrimeTsMin, qP-(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0)

[0436] Furthermore, in another embodiment, the quantization parameter qP may be derived as follows.

[0437] First, when the value of cIdx is 0, qP and ACT Qp offset can be derived as shown in the following equation.

[0438] [Formula 30]

[0439] qP=QP'Y

[0440] ActQpOffset=5

[0441] Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, the qP and ACTQp offsets may be derived as shown below.

[0442] [Equation 31]

[0443] qP=Qp'CbCr

[0444] ActQpOffset=5

[0445] Alternatively, when the value of cIdx is 1, qP and ACT Qp offset may be derived as shown below.

[0446] [Formula 32]

[0447] qP=Qp'Cb

[0448] ActQpOffset=5

[0449] Alternatively, when the value of cIdx is 2, qP and ACT Qp offset can be derived as shown in the following equation.

[0450] [Formula 33]

[0451] qP=Qp'Cr

[0452] ActQpOffset=3

[0453] The quantization parameter qP may be updated as follows.

[0454] When the value of transform_skip_flag[xTbY][yTbY][cIdx] is 0, qP can be derived as shown in the following equation.

[0455] [Equation 34]

[0456] qP=Max(0, qP-(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset:0))

[0457] Alternatively, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1, qP may be derived as shown in the following equation.

[0458] [Formula 35]

[0459] qP=Max(0,Max(QpPrimeTsMin,qP)-(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0))

[0460] In another embodiment, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1, the value of QpPrimeTsMin instead of 0 may be used to clip qP, as shown in the following equation.

[0461] [Equation 36]

[0462] qP=Max(QpPrimeTsMin, qP-(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0)

[0463] Furthermore, in yet another embodiment, the quantization parameter qP may be derived as follows.

[0464] First, when the value of cIdx is 0, qP and ACT Qp offset can be derived as shown in the following equation.

[0465] [Formula 37]

[0466] qP=Qp'Y

[0467] ActQpOffset=-5

[0468] Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, the qP and ACTQp offsets may be derived as shown below.

[0469] [Equation 38]

[0470] qP=Qp'CbCr

[0471] ActQpOffset=-5

[0472] Alternatively, when the value of cIdx is 1, qP and ACT Qp offset may be derived as shown below.

[0473] [Equation 39]

[0474] qP=Qp'Cb

[0475] ActQpOffset=-5

[0476] Alternatively, when the value of cIdx is 2, qP and ACT Qp offset can be derived as shown in the following equation.

[0477] [Formula 40]

[0478] qP=Qp' Cr

[0479] ActQpOffset=-3

[0480] The quantization parameter qP may be updated as follows.

[0481] When the value of transform_skip_flag[xTbY][yTbY][cIdx] is 0, qP can be derived as shown in the following equation.

[0482] [Formula 41]

[0483] qP=Max(0, qP+(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0))

[0484] Alternatively, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1, qP may be derived as shown in the following equation.

[0485] [Formula 42]

[0486] qP=Max(0,Max(QpPrimeTsMin,qP)+(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0))

[0487] Furthermore, in another embodiment, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1, as shown in the following equation, the value of QpPrimeTsMin instead of 0 may be used to trim qP.

[0488] [Formula 43]

[0489] qP=Max(QpPrimeTsMin, qP+(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0)

[0490] In the above description, Y, Cb, and Cr may represent three color components. For example, in ACT transformation, Y may correspond to C0. Cb may correspond to C1 or Cg. In addition, Cr may correspond to C2 or Co.

[0491] In addition, the ACTQpOffset values ​​-5, -5, and -3 of the three color components may be replaced with other values ​​or other variables.

[0492] Implementation 2: Signaling of QP offset adjustment in ACT

[0493] In the above embodiment, the ACT QP offset adjustment is fixed to -5, -5 and -3 for Y, Cg and Co components. In this embodiment, in order to provide greater flexibility to the ACTQP adjustment offset, a method of signaling the ACTQP offset will be described. The ACT QP offset can be signaled as a parameter in the PPS.

[0494] In an implementation, qp_offset may be based on Fig.36 The syntax table of is signaled. Its syntax elements are as follows.

[0495] The syntax element pps_act_qp_offsets_present_flag may indicate whether a syntax element related to ACT QP offset is present in PPS. For example, pps_act_qp_offsets_present_flag may indicate whether syntax elements pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset described later are signaled as PPS.

[0496] For example, a first value (eg, 0) of pps_act_qp_offsets_present_flag may indicate that pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset are not signaled through the PPS syntax table.

[0497] A second value (eg, 1) of pps_act_qp_offsets_present_flag may indicate that pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset are signaled through the PPS syntax table.

[0498] When pps_act_qp_offsets_present_flag is not provided from the bitstream, pps_act_qp_offsets_present_flag may be derived as a first value (e.g., 0). For example, when a flag indicating whether ACT is applied (e.g., sps_act_enabled_flag signaled in SPS) has a first value (e.g., 0) indicating that ACT is not applied, pps_act_qp_offsets_present_flag may be forced to have a first value (e.g., 0).

[0499] When the value of the syntax element cu_act_enabled_flag is the second value (e.g., 1) indicating that ACT is applied to the current coding basis, the syntax elements pps_act_y_qp_offset_plus5, pps_act_cb_qp_offset_plus5, and pps_act_cr_qp_offset_plus3 may be used to determine the offsets of the quantization parameter values ​​qP applied to the luma Cb and Cr components, respectively. When the values ​​of pps_act_y_qp_offset_plus5, pps_act_cb_qp_offset_plus5, and pps_act_cr_qp_offset_plus3 do not exist in the bitstream, each value may be set to 0.

[0500] According to the syntax element, the value of the variable PpsActQpOffsetY may be determined as pps_act_y_qp_offset_plus5 - 5. The value of the variable PpsActQpOffsetCb may be determined as pps_act_cb_qp_offset_plus5 - 5. In addition, the value of the variable PpsActQpOffsetCr may be determined as pps_act_cb_qp_offset_plus3 - 3.

[0501] Herein, ACT is not an orthogonal transform, so 5, 5, and 3 may be applied as constant offset values ​​to be subtracted. In an embodiment, for bitstream matching, the values ​​of PpsActQpOffsetY, PpsActQpOffsetCb, and PpsActQpOffsetCr may have values ​​varying from -12 to 12. In addition, according to an embodiment, the Qp offset value may be replaced with other constant values ​​other than 5, 5, and 3 and used.

[0502] In another embodiment, a more flexible ACT_QP offset may be used to adjust the QP. In the following embodiments, an example of signaling the ACTQP offset in the bitstream is described. Therefore, the ACTQP offset may have a wider offset range. Therefore, the QP updated using the ACT QP offset is more likely to exceed the available range, so the upper and lower limits of the updated QP need to be trimmed (a more detailed embodiment will be described later in embodiments 6 and 7).

[0503] The variables PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, and PpsActQpOffsetCbCr indicating the ACT QP offset may be values ​​derived using the ACT QP offset signaled through the bitstream or a preset constant. For bitstream consistency, PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, and PpsActQpOffsetCbCr may have values ​​varying from -12 to +12.

[0504] When a fixed value is not used, the value of the QP offset is signaled and has a value ranging from -12 to 12. In addition to clipping the lower limit of the derived QP value to avoid the QP having negative values, the upper limit of the derived QP value also needs to be clipped.

[0505] To prevent the value of qP from having a negative value, the lowest value of qP may be forced to 0. Alternatively, the lowest value of qP may be set to a value determined by a signaled syntax element. For example, to signal the lowest value of qP when the transform skip mode is applied, a syntax element QpPrimeTsMin indicating the value of qP applied when the transform skip mode is applied may be used. The maximum value of qP may be limited to an available maximum value of qP (e.g., 63) or a maximum available qP value determined according to a signaled syntax element.

[0506] In the embodiment according to the above content, the quantization parameter qP can be derived as follows: First, when the value of cIdx is 0, qP and ACT Qp offset can be derived as shown in the following equation.

[0507] [Formula 44]

[0508] qP=Qp' Y

[0509] ActQpOffset=PpsActQpOffsetY

[0510] Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, the qP and ACTQp offsets may be derived as shown below.

[0511] [Formula 45]

[0512] qP=Qp'CbCr

[0513] ActQpOffset=PpsActQpOffsetCbCr

[0514] Alternatively, when the value of cIdx is 1, qP and ACT Qp offset may be derived as shown in the following equation.

[0515] [Formula 46]

[0516] qP=Qp'Cb

[0517] ActQpOffset=PpsActQpOffsetCb

[0518] Alternatively, when the value of cIdx is 2, qP and ACT Qp offset can be derived as shown in the following equation.

[0519] [Formula 47]

[0520] qP=Qp'Cr

[0521] ActQpOffset=PpsActQpOffsetCr

[0522] In an embodiment, the quantization parameter qP may be updated as follows: When the value of transform_skip_flag[xTbY][yTbY][cIdx] is 0, qP may be derived as shown in the following equation.

[0523] [Formula 48]

[0524] qP=Clip3(0,63,qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0525] Alternatively, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1, qP may be derived as shown in the following equation.

[0526] [Formula 49]

[0527] qP=Clip3(0, 63, Max(QpPrimeTsMin, qP)-(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0)

[0528] In another embodiment, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1, the value of QpPrimeTsMin instead of 0 may be used to clip the lowest value of qP as shown in the following equation.

[0529] [Formula 50]

[0530] The quantization parameter qP may be updated as follows.

[0531] When the value of transform_skip_flag[xTbY][yTbY][cIdx] is 0, qP can be derived as shown in the following equation.

[0532] qP=Clip3(0, 63, qP-(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0))

[0533] Alternatively, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1, qP may be derived as shown in the following equation.

[0534] qP=Clip3(QpPrimeTsMin, 63, qP-cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0)

[0535] In another embodiment, the quantization parameter qP may be updated as follows.

[0536] When the value of transform_skip_flag[xTbY][yTbY][cIdx] is 0, qP can be derived as shown in the following equation.

[0537] [Formula 51]

[0538] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0))

[0539] Alternatively, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1, qP can be derived as shown in the following equation.

[0540] [Formula 52]

[0541] qP=Clip3(0, 63+QpBdOffset, Max(QpPrimeTsMin, qP)+(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0)

[0542] In another embodiment, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1, the value of QpPrimeTsMin instead of 0 may be used to clip the lowest value of qP as shown in the following equation.

[0543] [Formula 53]

[0544] The quantization parameter qP may be updated as follows.

[0545] When the value of transform_skip_flag[xTbY][yTbY][cIdx] is 0, qP can be derived as shown in the following equation.

[0546] qP=Clip3(0,63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0))

[0547] Alternatively, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1, qP may be derived as shown in the following equation.

[0548] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0)

[0549] Embodiment 3: Method for allowing ACT when executing chroma BDPCM

[0550] In an embodiment, when BDPCM is applied to a luma component block, ACT may be applied to encode / decode the block. However, when BDPCM is applied to a chroma component block, ACT is restricted from being applied to encode / decode the block.

[0551] Furthermore, even when BDPCM is applied to a chroma component block, ACT is applied to the block, thereby improving encoding efficiency. Fig.37 FIG. 2 shows an embodiment of a syntax configuration in which ACT is applied even when BDPCM is applied to a chrominance component block. Fig.37 As shown, by removing the condition for obtaining the BDPCM syntax element of the chroma component according to the value of cu_act_enabled_flag indicating whether ACT is applied to the current encoding basis, its BDPCM syntax element can be obtained regardless of whether ACT is applied to the chroma component block, and BDCPM encoding can be performed accordingly.

[0552] Embodiment 4: Method of applying ACT even when encoding / decoding is performed using CCLM

[0553] Both CCLM and ACT aim to remove unnecessary overlaps between components. There are some overlapping parts between CCLM and ACT, but even if all of them are applied, it is impossible to completely remove the overlaps between components. Therefore, by applying CCLM and ACT together, the overlaps between components can be removed more effectively.

[0554] The following embodiments describe embodiments in which CCLM and ACT are applied together. When performing decoding, the decoding device may first apply CCLM and then apply ACT. When ACT is applied to both BDPCM and CCLM for chrominance components, the syntax table for signaling these may be as follows: Fig.38 is modified as shown. Therefore, Fig.38 As shown in the syntax table, among the restrictions for signaling syntax elements related to intra_bdpcm_chroma and ccm, if(!cu_act_enabled_flag) for signaling syntax elements according to whether ACT is not applied can be removed from the syntax table.

[0555] Embodiment 5: Method using flexible ACT Qp including combined CbCr

[0556] When the ACT mode is applied, the prediction residual can be converted from one color space (e.g., GBR or YCbCr) to the YCgCo color space. In addition, the residual of the transform basis can be encoded in the YCgCo color space. As an implementation of the ACT kernel transform (transform kernel) for color space transform, the following transform kernel as described above can be used.

[0557] [Formula 54]

[0558]

[0559] [Formula 55]

[0560]

[0561] As described in the above formula, the transformations of C0', C1' and C2' (in this article, C0'=Y, C1'=Cg, C2'=Co) are not normalized. For example, the value of the L2 norm is not 1. For example, the L2 norm of the transformation of a single component can have a value of about 0.6 for C0' and C1', and a value of about 0.7 for C2'. In this article, the L2 norm is a value obtained as the square root of the sum of the squares of the corresponding coefficients. For example, C0'=2 / 4*C0+1 / 4*C1+1 / 4*C2 can be calculated. Therefore, the norm of C0' can be calculated as the square root of (2 / 4*2 / 4+1 / 4*1 / 4+1 / 4*1 / 4). Therefore, this can be calculated as the square root of 6 / 16, and can be calculated to have a value of about 0.6.

[0562] When the normalization transform is not applied, the dynamic range of a single component is irregular, which will lead to the degradation of the encoding performance of general video compression systems.

[0563] In order to compensate for the dynamic range of the residual signal, a QP offset value for compensating for the dynamic range variation of a single transform component is transmitted so that QP adjustment can be performed. For example, this embodiment can be applied to a general QP adjustment control method for ACT transform and joint CbCr.

[0564] The individual color components are not encoded separately but are encoded together, so that the method for joint CbCr in the above-mentioned embodiment 3 can cause the dynamic range changes between the individual color components.

[0565] In the encoding and decoding methods according to the embodiment, the ACT QP offset adjustment may be fixed to -5, which may be equally applied to Y, Cg, and Co.

[0566] In an embodiment, to provide flexible Qp control for individual components and joint CbCr, different ACT Qp offsets may be allowed for Y, Cb, Cr and / or joint CbCr. The ACT Qp offset value may be determined based on component index and / or joint CbCr and / or joint CbCr mode.

[0567] To indicate the ACT Qp offset, ppsActQpOffsetY, ppsActQpOffsetCb, and ppsActQpOffsetCr may be used. In addition, ppsActQpOffsetCbCr may be used for the ACTQP offset for joint CbCr mode 2 with CBF, where all Cb and Cr components have non-zero values. These values ​​(e.g., ppsActQpOffsetY, ppsActQpOffsetCb, ppsActQpOffsetCr, and ppsActQpOffsetCbCr) may be predetermined to predetermined values ​​or signaled through the bitstream. The ACT QP offset for the joint CbCr mode may be set in another method or to another value.

[0568] In an embodiment, ACT Qp offsets of -5, -5, and -3 may be used for Y, Cb, and Cr, and an ACT Qp offset of -4 may be used for joint CbCr.

[0569] In another embodiment, ACT Qp offsets of -5, -4, and -3 may be used for Y, Cb, and Cr, and an ACT Qp offset of -3 may be used for joint-CbCr mode, where the value of tu_cbf_cb is not zero.

[0570] In yet another embodiment, the ACT QP offset of joint CbCr mode 2 may have its own offset value. For another joint CbCr mode, the ACT QP offset may use the offset of the corresponding component. For example, the quantization parameter qP may be determined as follows. First, when the value of cIdx is 0, qP and ACT Qp offset may be derived as shown in the following equation.

[0571] [Formula 56]

[0572] qP=Qp' Y

[0573] ActQpOffset=ppsActQpOffsetY

[0574] Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, the qP and ACTQp offsets may be derived as shown below.

[0575] [Formula 57]

[0576] qP=Qp' CbCr

[0577] ActQpOffset=ppsActQpOffsetCbCr

[0578] Alternatively, when the value of cIdx is 1, qP and ACT Qp offset may be derived as shown below.

[0579] [Formula 58]

[0580] qP=Qp' Cb

[0581] ActQpOffset=ppsActQpOffsetCb

[0582] Alternatively, when the value of cIdx is 2, qP and ACT Qp offset can be derived as shown in the following equation.

[0583] [Formula 59]

[0584] qP=Qp' Cr

[0585] ActQpOffset=ppsActQpOffsetCr

[0586] In an embodiment, the quantization parameter qP may be updated as follows.

[0587] When the value of transform_skip_flag[xTbY][yTbY][cIdx] is 0, qP can be derived as shown in the following equation.

[0588] [Formula 60]

[0589] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0))

[0590] Alternatively, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1, qP can be derived as shown in the following equation.

[0591] [Formula 61]

[0592] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0)

[0593] In another embodiment, for a joint CbCr mode with tu_cbf_cb!=0 (e.g., in the case of mode 1 and mode 2), ppsActQpOffsetCb may be used to determine the offset of the joint CbCr. Alternatively, for a joint CbCr mode with tu_cbf_cb==0 (e.g., in the case of mode 3), ppsActQpOffsetCr may be used to determine the offset of the joint CbCr. For example, the above embodiment may be modified and applied as follows.

[0594] The quantization parameter qP may be updated as follows: First, when the value of cIdx is 0, qP and ACTQp offset may be derived as shown in the following equation.

[0595] [Formula 62]

[0596] qP=Qp' Y

[0597] ActQpOffset=ppsActQpOffsetY

[0598] Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, qP can be derived as shown in the following equation.

[0599] [Formula 63]

[0600] qP=Qp' CbCr

[0601] Alternatively, when the value of cIdx is 1, qP and ACT Qp offset may be derived as shown below.

[0602] [Formula 64]

[0603] qP=Qp' Cb

[0604] ActQpOffset=ppsActQpOffsetCb

[0605] Alternatively, when the value of cIdx is 2, qP and ACT Qp offset can be derived as shown in the following equation.

[0606] [Formula 65]

[0607] qP=Qp' Cr

[0608] ActQpOffset=ppsActQpOffsetCr

[0609] When the value of cIdx is not 0 and the value of TuCResMode[xTbY][yTbY] is not 0, the ACT Qp offset of the joint CbCr mode can be determined according to the following pseudo code.

[0610] [Formula 66]

[0611] if(TuCResMode[xTbY][yTbY]is euqal to 1 or 2)

[0612] ActQpOffset=ppsActQpOffsetCb;

[0613] else

[0614] ActQpOffset=ppsActQpOffsetCr;

[0615] In an embodiment, the quantization parameter qP may be updated as follows: When the value of transform_skip_flag[xTbY][yTbY][cIdx] is 0, qP may be derived as shown in the following equation.

[0616] [Formula 67]

[0617] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0))

[0618] Alternatively, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1, qP may be derived as shown in the following equation.

[0619] [Formula 68]

[0620] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0)

[0621] In another embodiment, regardless of the joint CbCr mode, ppsActQpOffsetY is used when the component index is Y, ppsActQpOffsetCb is used when the component index is Cb, and ppsActQpOffsetCr is used when the component index is Cr, from which qP can be derived. For example, the quantization parameter qP can be derived as follows.

[0622] First, when the value of cIdx is 0, qP and ACTQp offsets can be derived as shown in the following equations.

[0623] [Formula 69]

[0624] qP=Qp'Y

[0625] ActQpOffset=ppsActQpOffsetY

[0626] Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, the qP and ACTQp offsets may be derived as shown below.

[0627] [Formula 70]

[0628] qP=Qp' CbCr

[0629] ActQpOffset=(cIdx==1)? ppsActQpOffsetCb:ppsActQpOffsetCr

[0630] Alternatively, when the value of cIdx is 1, qP and ACT Qp offset may be derived as shown below.

[0631] [Formula 71]

[0632] qP=Qp' Cb

[0633] ActQpOffset=ppsActQpOffsetCb

[0634] Alternatively, when the value of cIdx is 2, qP and ACT Qp offset can be derived as shown in the following equation.

[0635] [Formula 72]

[0636] qP=Qp'Cr

[0637] ActQpOffset=ppsActQpOffsetCr

[0638] The quantization parameter qP may be updated as follows.

[0639] When the value of transform_skip_flag[xTbY][yTbY][cIdx] is 0, qP can be derived as shown in the following equation.

[0640] [Formula 73]

[0641] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0))

[0642] Alternatively, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1, qP may be derived as shown in the following equation.

[0643] [Formula 74]

[0644] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY]? ActQpOffset: 0)

[0645] Embodiment 6: Method for signaling ACT Op offset including joint CbCr

[0646] In the following, an example of signaling the ACTQP offset through the bitstream to provide more flexibility is described. The ACTQP offset can be signaled through the SPS, PPS, picture header, slice header, or other types of header sets. The ACT Qp offset of the joint CbCr can be signaled separately or can be derived from the ACT QP offsets of Y, Cb, and Cr.

[0647] Without loss of generality, Fig.39 An example of a syntax table for signaling an ACT Qp offset in a PPS is shown. Fig.39 In an embodiment, an ACT Qp offset may be signaled for the joint CbCr. Fig.39 The syntax elements indicated in the syntax table.

[0648] The syntax element pps_act_qp_offsets_present_flag may indicate whether a syntax element related to ACT QP offset is present in PPS. For example, pps_act_qp_offsets_present_flag may indicate whether syntax elements pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4 to be described later are signaled as PPS.

[0649] For example, a first value (eg, 0) of pps_act_qp_offsets_present_flag may indicate that pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4 are not signaled through the PPS syntax table.

[0650] A second value (eg, 1) of pps_act_qp_offsets_present_flag may indicate that pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4 are signaled through the PPS syntax table.

[0651] When pps_act_qp_offsets_present_flag is not provided from the bitstream, pps_act_qp_offsets_present_flag may be derived as a first value (e.g., 0). For example, when a flag indicating whether ACT is applied (e.g., sps_act_enabled_flag signaled in SPS) has a first value (e.g., 0) indicating that ACT is not applied, pps_act_qp_offsets_present_flag may be forced to have a first value (e.g., 0).

[0652] When the value of the syntax element cu_act_enabled_flag is the second value (e.g., 1) indicating that ACT is applied to the current coding basis, the syntax elements pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4 may be used to determine the offset of the quantization parameter value qP applied to the luma component, the Cb component, the Cr component, and the joint CbCr component, respectively. When the values ​​of pps_act_y_qp_offset__plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4 do not exist in the bitstream, each value may be set to 0.

[0653] According to the syntax elements, the values ​​of variables PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr and PpsActQpOffsetCbCr may be determined as shown in the following equations.

[0654] [Formula 75]

[0655] PpsActQpOffsetY=pps_act_y_qp_offset_plusX1-X1

[0656] PpsActQpOffsetCb=pps_act_cb_qp_offset_plusX2-X2

[0657] PpsActQpOffsetCr=pps_act_cr_qp_offset_plusX3-X3

[0658] PpsActQpOffsetCbCr=pps_act_cbcr_qp_offset_plusX4-X4

[0659] Herein, X1, X2, X3, and X4 may indicate predetermined constant values. These may be the same value or different values, or only some may have the same value.

[0660] In an embodiment, for bitstream matching, the values ​​of PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, and PpsActQpOffsetCbCr may be restricted to have values ​​ranging from -12 to 12.

[0661] According to the determination of the variables, the quantization parameter qP can be determined as follows: First, when the value of cIdx is 0, qP and ACT Qp offset can be derived as shown in the following equation.

[0662] [Formula 76]

[0663] qP=Qp' Y

[0664] ActQpOffset=ppsActQpOffsetY

[0665] Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, the qP and ACTQp offsets may be derived as shown below.

[0666] [Formula 77]

[0667] qP=Qp' CbCr

[0668] ActQpOffset=(cIdx==1)? ppsActQpOffsetCb:ppsActQpOffsetCr

[0669] Alternatively, when the value of cIdx is 1, qP and ACT Qp offset may be derived as shown below.

[0670] [Formula 78]

[0671] qP=Qp' Cb

[0672] ActQpOffset=PpsActQpOffsetCb

[0673] Alternatively, when the value of cIdx is 2, qP and ACT Qp offset can be derived as shown in the following equation.

[0674] [Formula 79]

[0675] qP=Qp' Cr

[0676] ActQpOffset=PpsActQpOffsetCr

[0677] In another embodiment of signaling ACT Qp offsets, multiple ACT QP offsets may be signaled for different joint CbCr modes identified as Mode-A and Mode-B.

[0678] Joint CbCr mode A may refer to a joint CbCr mode including tu_cbf_cb having a non-zero value (e.g., mode 1 and mode 2 of Table 2 above). Joint CbCr mode B may refer to a joint CbCr mode including tu_cbf_cb having a value of 0 (e.g., mode 3 of Table 2 above). Fig.40 The corresponding changed syntax table is shown in Fig.40 The syntax elements indicated in the syntax table of

[0679] When the value of the syntax element cu_act_enabled_flag is the second value (e.g., 1) indicating that ACT is applied to the current coding basis, the syntax elements pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, pps_act_cbcr_qp_offset_modeA_plusX4, and pps_act_cbcr_qp_offset_modeB_plusX5 may be used to determine offsets of the quantization parameter values ​​qP applied to the luma component, Cb component, Cr component, and joint CbCr component, respectively. When the values ​​of pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, pps_act_cbcr_qp_offset_modeA_plusX4, and pps_act_cbcr_qp_offset_modeB_plusX5 do not exist in the bitstream, each value may be set to 0.

[0680] According to the syntax elements, the values ​​of variables PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, PpsActQpOffsetCbCrModeA and PpsActQpOffsetCbCrModeB may be determined as shown in the following equations.

[0681] [Formula 80]

[0682] PpsActQpOffsetY=pps_act_y_qp_offset_plusX1-X1

[0683] PpsActQpOffsetCb=pps_act_cb_qp_offset_plusX2-X2

[0684] PpsActQpOffsetCr=pps_act_cr_qp_offset_plusX3-X3

[0685] PpsActQpOffsetCbCrModeA=pps_act_cbcr_qp_offset_modeA_plusX4-X4

[0686] PpsActQpOffsetCbCrModeB=pps_act_cbcr_qp_offset_modeB_plusX5-X5

[0687] Herein, X1, X2, X3, X4, and X5 may indicate predetermined constant values. These may be the same value or different values, or only some may have the same value. In an embodiment, for bitstream matching, the values ​​of PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, PpsActQpOffsetCbCrModeA, and PpsActQpOffsetCbCrModeB may be restricted to have values ​​ranging from -12 to 12.

[0688] According to the determination of the variables, the quantization parameter qP can be determined as follows: First, when the value of cIdx is 0, qP and ACT Qp offset can be derived as shown in the following equation.

[0689] [Formula 81]

[0690] qP=Qp' Y

[0691] ActQpOffset=PpsActQpOffsetY

[0692] Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, qP can be derived as shown in the following equation.

[0693] [Formula 82]

[0694] qP=Qp' CbCr

[0695] Alternatively, when the value of cIdx is 1, qP and ACT Qp offset may be derived as shown below.

[0696] [Formula 83]

[0697] qP=Qp'Cb

[0698] ActQpOffset=PpsActQpOffsetCb

[0699] Alternatively, when the value of cIdx is 2, qP and ACT Qp offset can be derived as shown in the following equation.

[0700] [Formula 84]

[0701] qP=Qp' Cr

[0702] ActQpOffset=PpsActQpOffsetCr

[0703] In addition, when the value of cIdx is not 0 and the value of TuCResMode[xTbY][yTbY] is not 0, the ACT Qp offset can be derived as shown in the following equation.

[0704] [Formula 85]

[0705] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])? PpsActQpOffsetCbCrModeA:PpsActQpOffsetCbCrModeB

[0706] Furthermore, in another embodiment, when the value of TuCResMode[xTbY][yTbY] is 2, ActQpOffset may be derived as shown in the following equation.

[0707] [Formula 86]

[0708] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])? (PPsQpOffsetCbCrModeA+slice_act_CbCr_qp_offset_ModeA): (PPsQpOffsetCbCrModeB+slice_act_CbCr_qp_offset_ModeB)

[0709] In another embodiment of signaling an ACT Qp offset, such as in Fig.41 As shown in the syntax table of , only the ACT QP offsets for Y, Cb, and Cr may be signaled. The ACT QP offset for joint CbCr may be derived from PpsActQpOffsetY, PpsActQpOffsetCb, and / or PpsActQpOffsetCr.

[0710] In an embodiment, the ACT Qp offset of CbCr may be set to the value of PpsActQpOffsetCb. In another embodiment, the ACT Qp offset of CbCr may be set to the same value as PpsActQpOffsetCb in the case of joint CbCr mode where tu_cbf_cb has a non-zero value, or may be set to the same value as PpsActQpOffsetCr in the case of joint CbCr mode where the value of tu_cbf_cb is 0. Alternatively, the opposite may be set.

[0711] Fig.41 is a diagram showing another embodiment of a syntax table for signaling an ACTQp offset in a PPS. Fig.41 The quantization parameter qP can be determined as follows: First, when the value of cIdx is 0, qP and ACT Qp offset can be derived as shown in the following formula.

[0712] [Formula 87]

[0713] qP=Qp' Y

[0714] ActQpOffset=PpsActQpOffsetY

[0715] Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, the qP and ACTQp offsets may be derived as shown below.

[0716] [Formula 88]

[0717] qP=QP' CbCr

[0718] ActQpOffset=(cIdx==1)? PpsActQpOffsetCb:PpsActQpOffsetCr

[0719] Furthermore, in another embodiment, the value of ActQpOffset may be determined as follows.

[0720] [Formula 89]

[0721] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])? PpsActQpOffsetCb:PpsActQpOffsetCr

[0722] Alternatively, when the value of cIdx is 1, qP and ACT Qp offset may be derived as shown below.

[0723] [Formula 90]

[0724] qP=Qp' Cb

[0725] ActQpOffset=PpsActQpOffsetCb

[0726] Alternatively, when the value of cIdx is 2, qP and ACT Qp offset can be derived as shown in the following equation.

[0727] [Formula 91]

[0728] qP=Qp' Cr

[0729] ActQpOffset=PpsActQpOffsetCr

[0730] Embodiment 7: Signaling ACTQp offset at multiple levels

[0731] In an embodiment, the ACT QP offset may be signaled at multiple levels. In addition to signaling the ACT QP offset at one level such as the PPS (as in Embodiment 6 above), the ACT QP offset may be signaled at a lower level (e.g., a slice header, a picture header, or other types of headers suitable for Qp control).

[0732] Hereinafter, two embodiments will be described. Fig.42 and Fig.43 An example of signaling the ACT QP offset through the slice header and the picture header is shown. In this way, the ACT QP offset can be signaled at multiple levels.

[0733] In the following, the Fig.42 and Fig.43 The syntax element pps_slice_act_qp_offsets_present_flag may indicate whether syntax elements slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset, which will be described later, are present in the slice header.

[0734] For example, a first value (eg, 0) of pps_slice_act_qp_offsets_present_flag may indicate that slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset do not exist in the slice header.

[0735] For example, a second value (eg, 1) of pps_slice_act_qp_offsets_present_flag may indicate that slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset are present in the slice header.

[0736] The syntax elements slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset may indicate offsets of quantization parameter values ​​qP for luma, Cb, Cr, and joint CbCr components, respectively. The values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset may be restricted to have values ​​varying from -12 to 12. When the values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_offset do not exist in the bitstream, each value may be set to 0. The values ​​of PpsActQpOffsetY+slice_act_y_qp_offset, PpsActQpOffsetCb+slice_act_cb_qp_offset, PpsActQpOffsetCr+slice_act_cr_qp_offset, and PpsActQpOffsetCbCr+slice_act_cbcr_qp_offset may also be restricted to have values ​​varying from -12 to 12.

[0737] Various modified implementations of signaling the ACT QP offset of joint CbCr at the PPS level may be applied. For example, one QP offset may be signaled for joint CbCr, multiple ACT Qp offsets may be signaled for joint CbCr of different modes, or without signaling the ACT Qp offset for joint CbCr, a method of deriving it by using ACTQpOffsets of Y, Cb and Cr and / or joint CbCr modes may be applied when performing signaling through a slice header.

[0738] Fig.44 and Fig.45 Two modified embodiments are shown. Fig.44 An implementation in which the ACTQp offset is signaled in the slice header is shown. Fig.45 Another embodiment of signaling the ACT Qp offset in the slice header is shown. Fig.45 In the embodiment, only the ACT Qp offsets of Y, Cb and Cr may be signaled, and the slice-level ACT QP offset of joint-CbCr may be derived from slice_act_y_qp_offset, slice_act_cb_qp_offset and / or slice_act_cr_qp_offset. This may be determined based on the mode type of joint-CbCr. In an embodiment, the slice-level ACTQp offset of CbCr may be set to the same value as slice_act_cb_qp_offset. In another embodiment, in the case of a joint-CbCr mode including tu_cbf_cb with a non-zero value, the slice-level ACT Qp offset of joint-CbCr may be set to the same value as slice_act_cb_qp_offset. In addition, in the case of a joint-CbCr mode including tu_cbf_cb with a value of 0, the slice-level ACT Qp offset of joint-CbCr may be set to the same value as slice_act_cr_qp_offset.

[0739] In addition, in another embodiment, the syntax elements may be signaled in a slice header or a picture header. To achieve this, encoding / decoding may be performed as follows.

[0740] - A flag pps_picture_slice_act_qp_offsets_present_flag indicating whether the ACT Qp offset is present in the picture header or the slice header may be signaled in the PPS.

[0741] -When ACT is applicable and the value of pps_picture_slice_act_qp_offsets_present_flag is a second value (e.g., 1), a flag pic_act_qp_offsets_present_flag indicating whether an ACT Qp offset is present in the picture header is signaled in the picture header. Herein, the second value (e.g., 1) of pic_act_qp_offsets_present_flag may indicate that the ACT Qp offsets of all slices of the picture corresponding to the picture header are provided in the picture header.

[0742] - A first value (e.g., 0) of pic_act_qp_offsets_present_flag may indicate that the ACT Qp offsets of all slices of the picture corresponding to the picture header are not provided in the picture header. For example, when ACT is applicable and the value of pps_picture_slice_act_qp_offsets_present_flag is the second value (e.g., 1) and the value of pic_act_qp_offsets_present_flag is the first value (e.g., 0), the ACT Qp offsets of the slice may be provided in the slice header.

[0743] Fig.46 is a diagram of a syntax table of a PPS that signals pps_pic_slice_act_qp_offsets_present_flag. The syntax element pps_pic_slice_act_qp_offsets_present_flag may indicate whether an ACT Qp offset is provided in a picture header and / or a slice header. For example, a first value (e.g., 0) of pps_pic_slice_act_qp_offsets_present_flag may indicate that an ACT Qp offset is not provided in a picture header and a slice header. A second value (e.g., 1) of pps_pic_slice_act_qp_offsets_present_flag may indicate that an ACT Qp offset is provided in a picture header or a slice header. When pps_pic_slice_act_qp_offsets_present_flag is not provided in a bitstream, the value of pps_pic_slice_act_qp_offsets_present_flag may be determined to be a first value (e.g., 0).

[0744] Fig.47 is a diagram illustrating a syntax table of a picture header for signaling an ACT Qp offset. The syntax element pic_act_qp_offsets_present_flag may indicate whether the ACT Qp offset is provided in the picture header. A first value of pic_act_qp_offsets_present_flag (e.g., 0) may indicate that the ACT Qp offset is not provided in the picture header but in the slice header. A second value of pic_act_qp_offsets_present_flag (e.g., 1) may indicate that the ACT Qp offset is provided in the picture header. When the value of pic_act_qp_offsets_present_flag is not provided in the bitstream, the value may be determined to be 0.

[0745] Fig.48 is a diagram showing a syntax table of a slice header for signaling an ACT Qp offset. Fig.48 In the syntax table of , the syntax elements slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset may indicate the offset of the quantization parameter value qP of the luma component, the Cb component, and the Cr component. The values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset may have values ​​varying from -12 to 12. In addition, the values ​​of PpsActQpOffsetY+slice_act_y_qp_offset, PpsActQpOffsetCb+slice_act_cb_qp_offset, and PpsActQpOffsetCr+slice_act_cr_qp_offset may also have values ​​varying from -12 to 12.

[0746] Furthermore, in a case where the values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset are not provided in a bitstream, when the value of pps_pic_slice_act_qp_offsets_present_flag is the first value (eg, 0), the values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, and slice_act_cr_qp_offset may be determined to be 0. Alternatively, when the value of pps_pic_slice_act_qp_offsets_present_flag is the second value (eg, 1), the values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, and slice_act_cb_act_cr_offset may be determined to be the same as pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset, respectively.

[0747] Furthermore, when an ACT Qp offset is present in both a slice header and a picture header, a final offset value for deriving a qP value may be determined as a value obtained by adding an offset value signaled in a PPS to an offset value signaled in a slice header or a picture header.

[0748] More specifically, in an embodiment, the quantization parameter qP may be determined as follows: First, when the value of cIdx is 0, qP and ACT Qp offset may be derived as shown in the following equation.

[0749] [Formula 92]

[0750] qP=Qp' Y

[0751] ActQpOffset=PPsQpOffsetY+slice_act_y_qp_offset

[0752] Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, qP and ACT Qp offset may be derived as shown in the following equations.

[0753] [Formula 93]

[0754] qP=Qp'CbCr

[0755] ActQpOffset=PPsQpOffsetCbCr+slice_act_CbCr_qp_offset

[0756] Alternatively, when the value of cIdx is 1, qP and ACT Qp offset may be derived as shown below.

[0757] [Formula 94]

[0758] qP=Qp'Cb

[0759] ActQpOffset=PpsActQpOffsetCb+slice_act_Cb_qp_offset

[0760] Alternatively, when the value of cIdx is 2, qP and ACT Qp offset may be derived as shown below.

[0761] [Formula 95]

[0762] qP=Qp'Cr

[0763] ActQpOffset=PpsActQpOffsetCr+slice_act_Cr_qp_offset

[0764] In another embodiment, when multiple ACT Qp offsets of a joint CbCr are signaled, the ActQpOffset of the joint CbCr may be determined as follows.

[0765] First, when the value of cIdx is 0, qP and ACTQp offsets can be derived as shown in the following equations.

[0766] [Formula 96]

[0767] qP=Qp'Y

[0768] ActQpOffset=PPsQpOffsetY+slice_act_y_qp_offset

[0769] Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, qP can be derived as shown in the following equation.

[0770] [Formula 97]

[0771] qP=Qp'CbCr

[0772] Alternatively, when the value of cIdx is 1, qP and ACT Qp offset may be derived as shown below.

[0773] [Formula 98]

[0774] qP=Qp'Cb

[0775] ActQpOffset=PpsActQpOffsetCb+slice_act_Cb_qp_offset

[0776] Alternatively, when the value of cIdx is 2, qP and ACT Qp offset may be derived as shown below.

[0777] [Formula 99]

[0778] qP=Qp'Cr

[0779] ActQpOffset=PpsActQpOffsetCr+slice_act_Ct_qp_offset

[0780] In addition, when the value of cIdx is not 0 and the value of TuCResMode[xTbY][yTbY] is not 0, the ACT Qp offset can be derived as shown in the following equation.

[0781] [Formula 100]

[0782] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])? (PPsQpOffsetCbCrModeA+slice_act_CbCr_qp_offset_ModeA): (PPsQpOffsetCbCrModeB+slice_act_CbCr_qp_offset_ModeB)

[0783] In yet another embodiment, when the ACT Qp offset of the joint CbCr is not provided, the ActQpOffset and qP of the Y, Cb and / or Cr components are determined, and the ACT Qp offset of the Y, Cb and / or Cr components may be used to determine the ActQpOffset of the joint CbCr as follows. For example, in the above embodiment, when the value of TuCResMode[xTbY][yTbY] associated with Equation 97 is 2, the calculation step of qP may be changed and performed as follows.

[0784] “Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, the qP and ACTQp offsets may be derived as shown below.

[0785] [Formula 101]

[0786] qP=Qp' CbCr

[0787] ActQpOffset=(cIdx==1])? (PPsQpOffsetCb+slice_act_Cb_qp_offset):(PPsQpOffsetCr+slice_act_Cr_qp_offset)”

[0788] In addition, in another embodiment, the value of ActQpOffset may be determined as follows.

[0789] [Formula 102]

[0790] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])? (PPsQpOffsetCb+slice_act_Cb_qp_offset):(PPsQpOffsetCr+slice_act_Cr_qp_offset)

[0791] Embodiment 8: Method for signaling multiple ACTQp offset sets

[0792] In this embodiment, a method of using the ACT Qp offset list will be described. To this end, the following process may be performed.

[0793] a) Multiple ACT Qp offset sets may be signaled in the form of a list within a parameter set (e.g., SPS or PPS). Each set in the list may include ACT Qp offsets for Y, Cb, Cr, and joint CbCr components. For simplicity, the ACT Qp offset list may be signaled in the same parameter set as the one used to signal the chroma Qp offset list.

[0794] b) The number of sets of ACT Qp offsets in the list may be the same as the number of chroma Qp offset sets signaled in the PPS.

[0795] c) As the ACT Qp offset for deriving the qP for each coding basis, an ACT Qp offset belonging to a list with an index of the chroma Qp offset for the coding basis (eg, cu_chroma_qp_offset_idx) may be used.

[0796] d) As an alternative to b) and c), the following operations may be performed.

[0797] - The number of sets of ACT Qp offsets in the list may be signaled. The number of sets of ACT Qp offsets in the list may be different from the number of sets of chroma Qp offsets.

[0798] - When ACT is applicable, an index indicating the index of the ACT Qp offset used for encoding basis may be signaled.

[0799] Without departing from the above concept, Fig.49 As shown, the syntax for signaling the ACT Qp offset list may be used. For example, when the value of cu_act_enabled_flag is 1, pps_act_y_qp_offset, pps_act_cb_qp_offset, pps_act_cr_qp_offset, and pps_act_cbcr_qp_offset may be used to determine the offsets of the quantization parameter values ​​qP to be applied to the luma component, Cb component, Cr component, and joint CbCr, respectively.

[0800] When the values ​​of pps_act_y_qp_offset, pps_act_cb_qp_offset, pps_act_cr_qp_offset, and pps_act_cbcr_qp_offset do not exist, each value may be derived to be 0.

[0801] When the value of cu_act_enabled_flag is the second value (e.g., 1) and the value of cu_chroma_qp_offset_flag is the second value (e.g., 1), act_y_qp_offset_list[i], act_cb_qp_offset_list[i], act_cr_qp_offset_list[i], and act_cbcr_qp_offset_list[i] may be used to determine the offsets of the quantization parameter values ​​qP applied to the luma component, the Cb component, the Cr component, and the joint CbCr component, respectively. When the values ​​of act_y_qp_offset_list[i], act_cb_qp_offset_list[i], act_cr_qp_offset_list[i], and act_cbcr_qp_offset_list[i] do not exist, each value may be derived to be 0.

[0802] In this embodiment, the quantization parameter qP may be determined as follows: First, when the value of cIdx is 0, qP and ACT Qp offset may be derived as shown in the following equation.

[0803] [Formula 103]

[0804] qP=Qp' Y

[0805] ActQpOffset=pps_act_y_qp_offset+(cu_chroma_qp_offset_flag)? act_y_qp_offset_list[cu_chroma_qp_offset_idx]: 0+slice_act_y_qp_offset

[0806] Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, qP and ACT Qp offset may be derived as shown in the following equations.

[0807] [Formula 104]

[0808] qP=Qp' CbCr

[0809] ActQpOffset=pps_act_cbcr_qp_offset+(cu_chroma_qp_offset_flag)? act_cbcr_qp_offset_list[cu_chroma_qp_offset_idx]: 0+slice_act_cbcr_qp_offset

[0810] Alternatively, when the value of cIdx is 1, qP and ACT Qp offset may be derived as shown below.

[0811] [Formula 105]

[0812] qP=Qp' Cb

[0813] ActQpOffset=pps_act_cb_qp_offset+(cu_chroma_qp_offset_flag)? act_cb_qp_offset_list[cu_chroma_qp_offset_idx]: 0+slice_act_cb_qp_offset

[0814] Alternatively, when the value of cIdx is 2, qP and ACT Qp offset may be derived as shown below.

[0815] [Equation 106]

[0816] qP=Qp' Cr

[0817] ActQpOffset=pps_act_cr_qp_offset+(cu_chroma_qp_offset_flag)? act_cr_qp_offset_list[cu_chroma_qp_offset_idx]: 0+slice_act_cr_qp_offset

[0818] Implementation 9: ACT color space conversion method applied to both lossless coding and lossy coding

[0819] The transformation between color spaces based on the above matrices for forward transformation and backward transformation can be organized as follows.

[0820] [Table 3]

[0821]

[0822] Since some values ​​are lost in the processing of Co and Cg, the transformation cannot be reconstructed to the original state. For example, when the sample values ​​of the RGB color space are converted to the sample values ​​of the YCgCo color space and the resulting values ​​are reversely converted back to the sample values ​​of the RGB color space, the values ​​of the original samples are not completely reconstructed. Therefore, the transformation according to Table 3 cannot be used for lossless coding. It is necessary to improve the color space transformation algorithm so that even when lossless coding is applied, no loss of sample values ​​occurs after the color space transformation. Implementation 9 and Implementation 10 describe color space transformation algorithms that can be applied to lossless coding as well as lossy coding.

[0823] In the embodiment described below, a method for performing ACT by using a color space transform that is reversible to the original state is described, which is applicable to lossy coding as well as lossless coding. This reversible color space transform can be applied to the above-mentioned encoding and decoding methods. The ACTQp offset can also be adjusted for the following color space transform. The color space transform according to the embodiment can be performed as shown in the following formula. For example, a forward transform from a GBR color space to a YCgCo color space can be performed according to the following formula.

[0824] [Equation 107]

[0825] Co=RB;

[0826] t=B+(Co>>1);

[0827] Cg=Gt;

[0828] Y=t+(Cg>>1);

[0829] In addition, backward conversion from the YCgCo color space to the GBR color space can be performed according to the following equation.

[0830] [Formula 108]

[0831] t=Y-(Cg>>1)

[0832] G=Cg+t

[0833] B=t-(Co>>1)

[0834] R=Co+B

[0835] According to the above formula, the transformation between the YCgCo color space and the RGB color space is reversible to the original state. That is, the color space transformation according to the equation supports perfect reconstruction. For example, even if a backward transformation is performed after a forward transformation, the sample value is maintained. Therefore, the color space transformation according to the equation can be called a reversible YCgCo-R color transformation. In this article, R can represent reversible, which means that reconstruction of the original state is achieved. Compared with existing transformations, a YCgCo-R transformation can be provided by increasing the bit depth of Cg and Co by 1. If this condition is met, other types of reversible transformations similar to the above-mentioned transformations can be used.

[0836] Because the transform as shown above has a different norm value from the above transforms, the ACT Qp offsets of Y, Cg, and Co can be adjusted to compensate for the dynamic range change caused by the color space transform.

[0837] It has been described that when the above-mentioned transformation is applied, the QCT Qp offset according to the embodiment may have a value of (-5, -5, -5) for Y, Cg, and Co. However, when the reversible transformation according to the present embodiment is applied, as the QCT Qp offset according to the embodiment, another value other than (-5, -5, -5) may be specified. For example, as the QCT Qp offset according to the embodiment, the values ​​(-5, 1, 3) for Y, Cg, and Co may be used.

[0838] In another embodiment, the ACT QP offset may be signaled through the bitstream as in Embodiment 6 or Embodiment 7 above.

[0839] For example, when the above YCgCo-R transform is used with an ACT QP offset of (-5, 1, 3), it is observed that there is no coding loss in the lossy coding environment (e.g., QP 22, 27, 32, 37), as shown in the figure below. In addition, it is observed that when ACT is applied, a further 5% coding performance is gained in achieving lossless coding.

[0840] [Table 4]

[0841] sequence Y U V RGB,TGM 1080p 0.0％ 0.2％ 0.1％ RGB,TGM 720p 0.2％ -0.1％ 0.1％ RGB, simulation -0.1％ -0.1％ 0.0％ RGB, mixed content -0.1％ 0.0％ -0.1％ RGB, camera shooting content -0.3％ 0.2％ -0.3％ All (RGB) 0.0％ 0.0％ 0.0％

[0842] The VVC specification for including an integrated ACT matrix can be described as in the following table.

[0843] [Table 5]

[0844]

[0845]

[0846] For example, the residual sample array r of size (nTbW)×(nTbH) can be updated as follows Y 、r Cb and r Cr .

[0847] [Equation 109]

[0848] Tmp=r Y [x][y]-(r Cb [x][y]>>1)

[0849] r Y [x][y]=tmp+r Cb [x][y]

[0850] r Cb [x][y]=tmp-(r Cr [x][y]>>1)

[0851] r Cr[x][y]=r Cb [x][y]+r Cr [x][y]

[0852] Embodiment 10: ACT execution method for performing multiple color transformations based on explicit signaling

[0853] In this embodiment, at least one color transform may be performed by the ACT. Which color transform to perform may be determined by a flag signaled in the bitstream. The flag may be signaled at multiple levels (e.g., SPS, PPS, picture header, and slice) or identifiable granularity.

[0854] In an embodiment, a predetermined flag may be signaled to indicate which ACT is applied. For example, when the value of the flag is 1, an ACT based on a reversible color transformation may be applied. When the value of the flag is 0, an ACT based on an irreversible color transformation may be applied.

[0855] In another embodiment, a predetermined flag of the ACT may be signaled to indicate which color transform is used. Fig.50 An example of the syntax signaled in SPS is shown in FIG. Fig.50 The syntax element sps_act_reversible_conversion may indicate whether to use a transformation that is irreversible to the original state. The first value of sps_act_reversible_conversion (e.g., 0) may indicate that ACT uses a transformation that is irreversible to the original state. The second value of sps_act_reversible_conversion (e.g., 1) may indicate that ACT uses a transformation that is reversible to the original state.

[0856] Therefore, the variable lossyCoding indicating whether to perform lossy coding can be set as follows.

[0857] [Formula 110]

[0858] lossyCoding=(!sps_act_reversible_conversion)

[0859] By using the lossyCoding flag, a pseudo code for a decoding apparatus to perform backward conversion from YCgCo to GBR in a decoding process can be expressed as follows.

[0860] [Formula 111]

[0861] If(sps_act_reversible_conversion==1)

[0862] {

[0863] / / YCgCo-R reversible conversion

[0864] t=Y-(Cg>>1)

[0865] G=Cg+t

[0866] B=t-(Co>>1)

[0867] R=Co+B

[0868] }

[0869] else{

[0870] t=Y-Cg

[0871] G=Y+Cg

[0872] B=t-Co

[0873] R=t+Co

[0874] }

[0875] Therefore, the VVC specification shown in Table 5 of Embodiment 9 may be modified as shown in the following table.

[0876] [Table 6]

[0877]

[0878] According to the above table, the residual update process using color space transformation can use the following parameters as input to the process. - variable nTbW indicating the block width

[0879] - variable nTbH indicating the block height

[0880] - by element r Y [x][y] is a (nTbW)×(nTbH) array of luminance residual samples. Y ,

[0881] - by element r Cb [x][y] is a (nTbW)×(nTbH) array of chroma residual samples. Cb ,

[0882] - by element r Cr [x][y] is a (nTbW)×(nTbH) array of chroma residual samples. Cr ,

[0883] The output of this processing is as follows.

[0884] - An update array r of size (nTbW) × (nTbH) of luma residual samples Y ,

[0885] - An update array r of size (nTbW) × (nTbH) of chroma residual samples Cb ,

[0886] - An update array r of size (nTbW) × (nTbH) of chroma residual samples Cr ,

[0887] By performing this process, the residual sample array r of size (nTbW)×(nTbH) is Y 、r Cb and r Cr It can be updated as follows.

[0888] First, when the value of sps_act_reversible_conversion is the second value (eg, 1), the residual sample array r of size (nTbW)×(nTbH) Y 、r Cb and r Cr It can be updated as shown below.

[0889] [Formula 112]

[0890] tmp=r Y [x][y]-(r Cb [x][y]>>1))

[0891] r Y [x][y]=tmp+r Cb [x][y])

[0892] r Cb [x][y]=tmp-(r Cr [x][y]>>1))

[0893] r Cr [x][y]=r Cb [x][y]+r Cr [x][y])

[0894] Otherwise (eg, when the value of sps_act_reversible_conversion is the first value (eg, 0)), the residual sample array r of size (nTbW)×(nTbH) is Y 、r Cb and r Cr It can be updated as shown below.

[0895] [Formula 113]

[0896] tmp=r Y [x][y]-r Cb [x][y]

[0897] r Y [x][y]=r Y [x][y]+r Cb [x][y]

[0898] r Cb [x][y]=tmp-r Cr [x][y]

[0899] r Cr [x][y]=tmp+r Cr [x][y]

[0900] There are some similarities between the YCgCo backward transform and the YCgCo-R backward transform. In the transform reversible to the original state, when Cg and Co are replaced by Cg'=Cg<<1 and Co'=Co<<1, this can be operated as a lossy backward transform. The following formula shows its implementation.

[0901] [Formula 114]

[0902] t=Y-(Cg'>>1)=Y-Cg

[0903] G=Cg'+t=Y+Cg

[0904] B=t-(Co'>>1)=t–Co=Y–Cg-Co

[0905] R=Co'+B=t+Co=Y–Cg+Co

[0906] Therefore, in an alternative embodiment, instead of maintaining two color transforms, only a transform that is reversible to the original state can be used. In the case of lossy encoding, the Cg and Co components can be scaled 1 / 2 times in the operation of the encoding device and can be scaled twice in the operation of the decoding device. This allows one integrated transform to be used even when supporting lossy and lossless cases. In addition, there is an additional advantage that the bit depth does not change even when lossy encoding is performed.

[0907] [Table 7]

[0908]

[0909]

[0910] In an implementation, the Fig.51 The syntax of to use a flag (e.g., actShiftFlag) that indicates which ACT shift is used. Fig.51In the syntax table of , the syntax element sps_act_shift_flag can indicate whether to apply a color component shift while applying ACT. For example, a first value of sps_act_shift_flag (e.g., 0) can indicate that a color component shift is not applied while applying ACT. A second value of sps_act_shift_flag (e.g., 1) can indicate that a color component shift is applied while applying ACT. The variable actShiftFlag can be set to the value of sps_act_shift_flag. Pseudocode for implementing a backward transform from YCgCo to GBR in a decoding device can be written using actShiftFlag as shown below.

[0911] [Table 8]

[0912]

[0913] Embodiment 11: ACT implementation method using derivation of transformation types to perform multiple color transformations In an embodiment, at least one color transform may be used when performing an ACT. In addition, which color transform type to use may be inferred based on other information in the bitstream.

[0914] In an embodiment, two types of ACT transformations may be used, including an ACT transformation that is reversible to the original state and an ACT transformation that is irreversible to the original state. The ACT transformation type may be derived from the transformation type. For example, as indicated by the variable tuIsTransformSkip, when the transformation type is transform skip, an ACT transformation that is reversible to the original state may be used. Otherwise (e.g., when the transformation type is not transform skip), an ACT transformation that is irreversible to the original state may be used. Two types of pseudo codes may be used.

[0915] [Table 9]

[0916]

[0917]

[0918] [Table 10]

[0919]

[0920] In another embodiment, the ACT transform type may be determined based on the QP value. When the Qp value is less than or equal to a predetermined threshold (e.g., QpPrimeTsMin), an ACT transform that is reversible to the original state may be used. Otherwise (e.g., when the Qp value exceeds the predetermined threshold), an irreversible ACT transform may be used.

[0921] Embodiment 12: QP derivation method using ACT QP offset

[0922] This embodiment is related to the above-mentioned embodiment 1 and embodiment 2. In embodiment 1 and embodiment 2, the derived Qp' has been described. Y , Qp' CbCr , Qp' Cb and Qp' Cr Included as QP. The methods described in Embodiments 1 and 2 correct the derived Qp value by using the ACT QP offset, and apply necessary clipping techniques so that the corrected QP value for transform coefficient scaling does not exceed the valid range.

[0923] This embodiment describes a method for deriving Qp' Y , Qp' CbCr , Qp' Cb and Qp' Cr The method of including the ACT QP offset in the QP derivation process of the transform coefficient scaling process. The QP derivation process already includes a predetermined pruning step so that the derived QP value does not exceed the valid range. Therefore, including the ACT QP offset in the QP derivation process can avoid the additional pruning step, simplify the overall QP derivation steps of the transform coefficient scaling process and ensure that the final QP does not exceed the valid range.

[0924] As described in the above-mentioned embodiments, the ACT QP offset may be pre-specified as a constant or may be signaled via a bitstream. Without deviating from consistency, the ACT QP offsets for Y, Cb, Cr, and CbCr may be described in the description described later as ppsActQpOffsetY, ppsActQpOffsetCb, ppsActQpOffsetCr, and ppsActQpOffsetCbCr. ppsActQpOffsetY, ppsActQpOffsetCb, ppsActQpOffsetCr, and ppsActQpOffsetCbCr may be constants or variables having values ​​varying from -M to N. In this document, each of M and N may be set to 12 in the case of lossy coding, and may be set to 0 in the case of lossless coding in the embodiments. In addition, at least one ACT QP offset may be derived from another ACT QP offset value. For example, ppsActQpOffsetCbCr may be set to the same value as ppsActQpOffsetCb or ppsActQpOffsetCr based on the joint CbCr mode.

[0925] The decoding process of QP derivation using the ACT QP offset may be performed as follows: First, in the case of a quantization parameter derivation process, the following parameters may be used for the process.

[0926] - luma coordinates (xCb, yCb) indicating the relative coordinates of the top left luma sample of the current coding block relative to the top left luma sample of the current picture,

[0927] - a variable cbWidth indicating the width of the current coding block based on each luma sample,

[0928] - a variable cbHeight indicating the height of the current coding block based on each luma sample,

[0929] - variable treeType, which indicates whether a single tree (SINGLE_TREE) or a dual tree is used to split the current coding tree node, and when a dual tree is used, whether the dual tree is a luma component dual tree (DAUL_TREE_LUMA) or a chroma component dual tree (DAUL_TREE_CHROMA)

[0930] In this quantization parameter derivation process, the brightness quantization parameter Qp' can be derived. Y and the chromaticity quantization parameter Qp' Cb , Qp' Cr and Qp' CbCr .

[0931] After that, the variable Qp can be derived as shown below Y .

[0932] [Formula 115]

[0933] Q Y =((qP Y_PRED +CuQpDeltaVal+64+2*QpBdOffset)%(64+QpBdOffset))-QpBdOffset

[0934] The brightness quantization parameter Qp' can be derived as shown below Y .

[0935] [Formula 116]

[0936] actQpOffsetY=cu_act_enabled_flag[xCb][yCb]? ppsActQpOffsetY:0

[0937] Qp' Y =Qp Y +QpBdOffset+actQpOffsetY

[0938] Qp' Y =Clip3(0,63+QpBdOffset,Qp' Y )

[0939] When the value of the variable ChromaArrayType indicating the type of the chroma array is not the first value (eg, 0) and treeType is SINGLE_TREE or DUAL_TREE_CHROMA, the following processing may be performed.

[0940] -When the value of treeType is DUAL_TREE_CHROMA, the variable Qp Y The value of may be set to the luma quantization parameter Qp corresponding to the luma coding basis covering the luma sample position (xCb+cbWidth / 2, yCb+cbHeight / 2) Y The value of is the same as the value of .

[0941] -The variable qP can be derived as shown below Cb ,qP Cr and qP CbCr .

[0942] [Formula 117]

[0943] Q Chroma =Clip3(-QpBdOffset,63,Qp Y ) (1118)

[0944] Q Cb =ChromaQpTable[0][qP Chroma ] (1119)

[0945] Q Cr =ChromaQpTable[1][qP Chroma ] (1120)

[0946] Q CbCr =ChromaQpTable[2][qP Chroma ] (1121)

[0947] -Color quantization parameter Qp' of Cb component and Cr component Cb and Qp' Cr And the chromaticity quantization parameter Qp' of the joint Cb-Cr coding CbCr It can be derived as shown below.

[0948] [Formula 118]

[0949] actQpOffsetCb=cu_act_enabled_flag[xCb][yCb]? ppsActQpOffsetCb:0

[0950] actQpOffsetCr=cu_act_enabled_flag[xCb][yCb]? ppsActQpOffsetCr:0

[0951] actQpOffsetCbCr=cu_act_enabled_flag[xCb][yCb]? ppsActQpOffsetCbCr:0

[0952] Qp' Cb =Clip3(-QpBdOffset,63,qP Cb +pps_cb_qp_offset+slice_cb_qp_offset+CuQpOffset Cb +

[0953] actQpOffsetCb)+QpBdOffset

[0954] Qp' Cr =Clip3(-QpBdOffset,63,qP Cr +pps_cr_qp_offset+slice_cr_qp_offset+CuQpOffset Cr +

[0955] actQpOffsetCr)+QpBdOffset

[0956] Qp' CbCr =Clip3(-QpBdOffset,63,qP CbCr +pps_joint_cbcr_qp_offset+slice_joint_cbcr_qp_offset+

[0957] CuQpOffset CbCr +actQpOffsetCbCr)+QpBdOffset

[0958] Next, a dequantization process of the transform coefficients may be performed and the following information may be used as input to this process.

[0959] - refers to the luminance coordinates (xTbY, yTbY) of the upper left corner sample of the current luminance transform block relative to the upper left corner luminance sample of the current picture,

[0960] - a variable nTbW indicating the width of the transform block,

[0961] - a variable nTbH indicating the height of the transform block,

[0962] - a variable predMode indicating the prediction mode of the encoding basis,

[0963] - variable cIdx indicating the color component of the current block

[0964] The output of the dequantization process of the transform coefficients may be an array d of scaled transform coefficients. In this context, the size of the array d may be (nTbW)×(nTbH). The individual elements constituting it may be identified as d[x][y].

[0965] When this process is performed, the quantization parameter qP can be derived as follows: When the value of cIdx is 0, qP can be derived as shown in the following equation.

[0966] [Formula 119]

[0967] qP=Qp' Y

[0968] Alternatively, when the value of TuCResMode[xTbY][yTbY] is 2, it can be derived as shown in the following formula.

[0969] [Formula 120]

[0970] qP=Qp' CbCr

[0971] Alternatively, when the value of cIdx is 1, qP can be derived as shown in the following equation.

[0972] [Formula 121]

[0973] qP=Qp' Cb

[0974] Alternatively, when the value of cIdx is 2, qP can be derived as shown in the following equation.

[0975] [Formula 122]

[0976] qP=Qp' Cr

[0977] The quantization parameter qP may be updated as follows. In addition, the variables rectNonTsFlag and bdShift may be derived as follows. When the value of transform_skip_flag[xTbY][yTbY][cIdx] is 0, the derivation may be performed as shown in the following equation.

[0978] [Formula 123]

[0979] rectNonTsFlag=((((Log2(nTbW)+Log2(nTbH))&1)==1)?1:0

[0980] bdShift=BitDepth+rectNonTsFlag+((Log2(nTbW)+Log2(nTbH)) / 2)-5+pic_dep_quant_enabled_flag

[0981] Alternatively, when the value of transform_skip_flag[xTbY][yTbY][cIdx] is 1 (eg, skipping the transformation of the current transform block), it may be derived as shown in the following equation.

[0982] [Formula 124]

[0983] qP=Max(QpPrimeTsMin,qP)

[0984] rectNonTsFlag=0

[0985] bdShift=10

[0986] Encoding and decoding methods

[0987] In the following, reference will be made to Fig.52 and Fig.53 An image encoding method performed by an image encoding device and an image decoding method performed by an image decoding device are described.

[0988] First, the operation of the decoding device will be described. The image decoding device according to the embodiment may include a memory and a processor. The decoding device may perform decoding according to the operation of the processor. For example, Fig.52 As shown, the decoding device may determine the current block in step S5210 by segmenting the image based on the segmentation information obtained from the bitstream. Next, the quantization parameter of the current block may be determined in step S5220. Next, the transform coefficient of the current block may be determined based on the quantization parameter in step S5230. In this article, the quantization parameter may be determined based on whether the color space transform is applied to the residual sample of the current block. In addition, the value of the quantization parameter may be determined to have a value within a range set from a predetermined upper limit to a predetermined lower limit. For example, the value of the quantization parameter may be determined by performing pruning on the quantization parameter to have a value within the range. The predetermined lower limit value of the quantization parameter may be 0. In addition, the predetermined upper limit value of the quantization parameter may be determined based on a predetermined constant value (e.g., 63). Alternatively, as described above, it may be determined based on a parameter (e.g., QpBdOffset) indicating a luminance and chrominance quantization parameter range offset. QpBdOffset may be preset as a predetermined constant, or may be obtained based on a syntax element determined in the encoding process. For example, QpBdOffset may be calculated by multiplying a predetermined constant by the value of a syntax element indicating the bit depth of luma or chroma samples.

[0989] The quantization parameter may be determined based on a quantization parameter offset determined based on a color component of the current block. For example, the quantization parameter may be determined by adding a quantization parameter offset to a quantization parameter determined based on a color component of the current block. When a color space transform is applied to a residual sample of the current block and the color component of the current block is a luminance component, the absolute value of the quantization parameter offset may be determined to be 5. In addition, when a color space transform is applied to a residual sample of the current block and the color component of the current block is a chroma Cb component, the absolute value of the quantization parameter offset may be determined to be 1. In addition, when a color space transform is applied to a residual sample of the current block and the color component of the current block is a chroma Cr component, the absolute value of the quantization parameter offset may be determined to be 3.

[0990] In addition, the residual samples of the current block may be determined based on the transform coefficients of the current block. In this article, the values ​​of the residual samples may be updated based on whether the color space transform is applied. For example, when the color space transform is applied, the values ​​of the residual samples may be updated by performing the color space transform on the residual samples using a color space transform function that is reversible to the values ​​of the original samples. For example, when the color space transform is applied, the values ​​of the residual samples may be updated based on the sample values ​​obtained by applying a right shift operation to the chrominance component residual sample values.

[0991] Hereinafter, the operation of the encoding device will be described. The image encoding device according to the embodiment may include a memory and a processor. The encoding device may perform encoding according to the operation of the processor in a manner corresponding to the decoding of the decoding device. For example, Fig.53 As shown, the encoding device may determine the current block by dividing the image in step S5310. Next, the encoding device may determine the quantization parameter of the current block in step S5320. Next, the encoding device may determine the transform coefficient of the current block based on the quantization parameter in step S5330. In this article, the determination of the quantization parameter may be performed based on whether the color space transform is applied to the residual sample of the current block. In addition, the value of the quantization parameter may be limited to have a value within a range set from a predetermined upper limit to a predetermined lower limit.

[0992] Herein, the encoding device may apply pruning to the quantization parameter so that the value of the quantization parameter has a value within a predetermined range. Herein, the lower limit value of the predetermined range may be 0. In addition, the upper limit value of the predetermined range may be the upper limit value of the brightness and chrominance quantization parameters. The upper limit values ​​of the brightness and chrominance quantization parameters may be determined based on the bit depth of the sample.

[0993] In addition, in order to determine the quantization parameter, the encoding device may determine the quantization parameter based on the color component of the current block, and may determine the quantization parameter offset based on the color component of the current block. In addition, the encoding device may reset the quantization parameter by using the quantization parameter offset to determine the quantization parameter.

[0994] Herein, the encoding device may add a quantization parameter offset to the quantization parameter, thereby resetting the quantization parameter.

[0995] Herein, when the color space transform is applied to the residual sample of the current block and the color component of the current block is a luminance component, the value of the quantization parameter offset may be determined to be -5. In addition, when the color space transform is applied to the residual sample of the current block and the color component of the current block is a chrominance Cb component, the value of the quantization parameter offset may be determined to be 1. In addition, when the color space transform is applied to the residual sample of the current block and the color component of the current block is a chrominance Cr component, the value of the quantization parameter offset may be determined to be 3.

[0996] In addition, the residual samples of the current block are determined based on the transform coefficients of the current block, and the values ​​of the residual samples can be updated based on whether the color space transform is applied. When the color space transform is applied, the values ​​of the residual samples can be updated based on the sample values ​​obtained by applying a right shift operation to the chrominance component residual sample values.

[0997] Application Implementation

[0998] Although the exemplary method of the present disclosure described above is represented as a series of operations for the sake of clarity of description, it is not intended to limit the order of executing the steps, and these steps can be performed simultaneously or in different orders when necessary. In order to implement the method according to the present disclosure, the steps described may also include other steps, may include the remaining steps except some steps, or may include other additional steps except some steps.

[0999] In the present disclosure, an image encoding device or an image decoding device that performs a predetermined operation (step) may perform an operation (step) of confirming an execution condition or situation of the corresponding operation (step). For example, if it is described that a predetermined operation is performed when a predetermined condition is met, the image encoding device or the image decoding device may perform the predetermined operation after determining whether the predetermined condition is met.

[1000] The various embodiments of the present disclosure are not a list of all possible combinations and are intended to describe representative aspects of the present disclosure, and matters described in the various embodiments may be applied independently or in combination of two or more.

[1001] Various embodiments of the present disclosure may be implemented in hardware, firmware, software or a combination thereof. In the case of implementing the present disclosure in hardware, the present disclosure may be implemented in an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a general purpose processor, a controller, a microcontroller, a microprocessor, etc.

[1002] In addition, the image decoding device and the image encoding device to which the embodiments of the present disclosure are applied may be included in a multimedia broadcast transmission and reception device, a mobile communication terminal, a home theater video device, a digital theater video device, a surveillance camera, a video chat device, a real-time communication device such as video communication, a mobile streaming device, a storage medium, a camera, a video on demand (VoD) service provider, an OTT video (over the top video) device, an Internet streaming service provider, a three-dimensional (3D) video device, a video phone video device, a medical video device, etc., and may be used to process a video signal or a data signal. For example, an OTT video device may include a game console, a Blu-ray player, an Internet access TV, a home theater system, a smart phone, a tablet PC, a digital video recorder (DVR), etc.

[1003] Fig.54 is a diagram showing a content streaming system to which an embodiment of the present disclosure can be applied.

[1004] like Fig.54 As shown in , a content streaming system to which the embodiments of the present disclosure are applied may mainly include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[1005] The encoding server compresses the content input from a multimedia input device such as a smart phone, camera, or camcorder into digital data to generate a bitstream and sends the bitstream to the streaming server. As another example, when a multimedia input device such as a smart phone, camera, or camcorder directly generates a bitstream, the encoding server can be omitted.

[1006] A bitstream may be generated by an image encoding method or an image encoding device to which an embodiment of the present disclosure is applied, and a streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[1007] The streaming server sends multimedia data to the user device based on the user's request through the network server, and the network server is used as a medium to inform the user of the service. When the user requests the required service from the network server, the network server can deliver it to the streaming server, and the streaming server can send the multimedia data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server is used to control the command / response between the devices in the content streaming system.

[1008] The streaming server can receive content from a media storage device and / or an encoding server. For example, when receiving content from an encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream within a predetermined time.

[1009] Examples of user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays), digital televisions, desktop computers, digital signage, etc.

[1010] Each server in the content streaming system may operate as a distributed server, in which case data received from the various servers may be distributed.

[1011] The scope of the present disclosure includes software or machine-executable commands (e.g., operating systems, applications, firmware, programs, etc.) for enabling operations of methods according to various embodiments to be performed on a device or computer, and non-transitory computer-readable media having such software or commands stored thereon and executable on a device or computer.

[1012] Industrial Applicability

[1013] The embodiments of the present disclosure may be used to encode or decode an image.

Claims

1. An image decoding method performed by an image decoding device, the image decoding method comprising the following steps: Determine a quantization parameter for the current block; deriving transform coefficients of the current block based on the quantization parameter; deriving a residual block based on the transform coefficients of the current block; as well as reconstructing the current block based on the residual block, Wherein, the quantization parameter is determined based on a quantization parameter offset, wherein the quantization parameter offset is derived based on a color space transform of residual samples of the current block and a color component of the current block, and wherein the value of the quantization parameter is clipped to be greater than or equal to a specific value, the specific value being set based on a transform skip flag of the current block, and The transform skip flag is obtained based on the color component of the current block.

2. The method according to claim 1, in, The value of the quantization parameter is clipped to be less than or equal to a predetermined larger value.

3. The method according to claim 1, in, The transform skip flag is information on the transform from the frequency domain to the spatial domain.

4. The method according to claim 3, in, The transform skip flag based on the current block indicates that the transform from the frequency domain to the spatial domain is determined based on another syntax element, and the specific value is set to 0.

5. The method according to claim 3, in, The transform skip flag based on the current block indicates skipping the transform from the frequency domain to the spatial domain, and the specific value is set based on a minimum value of the quantization parameter for a transform skip mode.

6. The method according to claim 1, in, Based on the fact that the color component of the current block is a chrominance Cr component and the color space transform for the residual sample of the current block is applied, the absolute value of the quantization parameter offset is set to 3.

7. The method according to claim 1, in, Based on the color component of the current block being a luminance component and the color space transform for the residual samples of the current block being applied, the absolute value of the quantization parameter offset is set to 5.

8. The method according to claim 1, in, Based on the color space transform applied to the residual sample of the current block, the difference between the value of the quantization parameter offset based on the color component of the current block being the chroma Cr component and the value of the quantization parameter offset based on the color component of the current block being the chroma Cb component is 2.

9. An image encoding method performed by an image encoding device, the image encoding method comprising the following steps: Derive the residual block of the current block; determining a quantization parameter for the current block; and encoding transform coefficients of the residual block for the current block based on the quantization parameter, Wherein, the quantization parameter is determined based on a quantization parameter offset, wherein the quantization parameter offset is derived based on a color space transform of residual samples of the current block and a color component of the current block, and wherein the value of the quantization parameter is clipped to be greater than or equal to a specific value, the specific value being set based on a transform for the current block, and Therein, the transformation is determined based on the color components of the current block.

10. The method according to claim 9, in, The value of the quantization parameter is clipped to be less than or equal to a predetermined larger value.

11. The method according to claim 9, in, The transform is from spatial domain to frequency domain and is encoded as a transform skip flag.

12. The method according to claim 9, in, The specific value is set to 0 or a minimum value of the quantization parameter for the frequency transform skip mode.

13. The method according to claim 9, in, Based on the fact that the color component of the current block is a chrominance Cr component and the color space transform for the residual sample of the current block is applied, the absolute value of the quantization parameter offset is set to 3.

14. A method for transmitting a bit stream generated by the image encoding method according to claim 9.

Images