Image encoding / decoding method and bitstream transmission method

Abstract

The invention relates to an image encoding / decoding method and a method for transmitting a bitstream. An image encoding / decoding method and apparatus are provided. An image decoding method performed by an image decoding apparatus according to the present disclosure may comprise the steps of: determining a prediction mode of a current block on the basis of prediction information obtained from a bitstream; obtaining a flag indicating whether to apply a color space transform to a residual sample of the current block based on the prediction mode of the current block; determining a quantization parameter of the current block based on the flag; and determining a transform coefficient of the current block based on the quantization parameter.

Description

[0001] This application is a divisional application of the invention patent application with the original application number 202080089263.8 (International application number: PCT / KR2020 / 014846, application date: October 28, 2020, invention name: Image encoding / decoding method and device using adaptive color transformation and method for sending bit stream). Technical Field

[0002] The present disclosure relates to an image encoding / decoding method and apparatus, and more particularly to an image encoding / decoding method and apparatus using adaptive color transform, and a method of transmitting a bit stream generated by the image encoding method / apparatus of the present disclosure. Background Art

[0003] Recently, the demand for high-resolution and high-quality images, such as high-definition (HD) images and ultra-high-definition (UHD) images, is increasing in various fields. As the resolution and quality of image data are improved, the amount of information or bit amount transmitted is relatively increased compared to existing image data. The increase in the amount of information or bit amount transmitted leads to an increase in transmission cost and storage cost.

[0004] Therefore, efficient image compression technology is needed to effectively transmit, store, and reproduce information about high-resolution and high-quality images. Summary of the invention

[0005] Technical issues

[0006] The present disclosure aims to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0007] In addition, the present disclosure is directed to providing an image encoding / decoding method and apparatus for improving encoding / decoding efficiency by performing adaptive color transformation.

[0008] In addition, the present disclosure is intended to provide a method of transmitting a bit stream generated by the image encoding method or apparatus according to the present disclosure.

[0009] In addition, the present disclosure is intended to provide a recording medium storing a bit stream generated by the image encoding method or apparatus according to the present disclosure.

[0010] In addition, the present disclosure is intended to provide a recording medium storing a bit stream received and decoded by the image decoding apparatus according to the present disclosure and used to reconstruct an image.

[0011] Those skilled in the art will understand that the technical objectives to be achieved by the present disclosure are not limited to the above-mentioned technical objectives, and other technical objectives not described herein will be clearly understood from the following description.

[0012] Technical Solution

[0013] According to one aspect of the present disclosure, there is provided an image decoding method performed by an image decoding device, the method comprising the following steps: determining a prediction mode of a current block based on prediction information obtained from a bitstream; obtaining a flag indicating whether to apply a color space transform to a residual sample of the current block based on the prediction mode of the current block; determining a quantization parameter of the current block based on the flag; and determining a transform coefficient of the current block based on the quantization parameter. Herein, the step of determining the quantization parameter may be performed by applying clipping to the quantization parameter so that the value of the quantization parameter has a value within a predetermined range.

[0014] In addition, according to an aspect of the present disclosure, there is provided an image decoding device including a memory and at least one processor, wherein the at least one processor is configured to: determine a prediction mode of a current block based on prediction information obtained from a bitstream; obtain a flag indicating whether to apply a color space transform to a residual sample of the current block based on the prediction mode of the current block; determine a quantization parameter of the current block based on the flag; and determine a transform coefficient of the current block based on the quantization parameter. Herein, the processor may be configured to determine the quantization parameter by applying pruning to the quantization parameter so that the value of the quantization parameter has a value within a predetermined range.

[0015] In addition, according to one aspect of the present disclosure, there is provided an image encoding method performed by an image encoding device, the method comprising the following steps: determining a prediction mode of a current block; determining a quantization parameter of the current block according to whether a color space transform is applied to a residual sample of the current block; determining a transform coefficient of the current block based on the quantization parameter; and encoding information indicating whether a color space transform is applied to the residual sample of the current block based on the prediction mode of the current block. Herein, the step of determining the quantization parameter may be performed by applying clipping to the quantization parameter so that the value of the quantization parameter has a value within a predetermined range.

[0016] In addition, according to another aspect of the present disclosure, a transmission method for transmitting a bit stream generated by the image encoding device or the image encoding method of the present disclosure is provided.

[0017] In addition, according to another aspect of the present disclosure, a computer-readable recording medium storing a bit stream generated by the image encoding method or the image encoding device of the present disclosure is provided.

[0018] The features described above in brief summary of the present disclosure are merely exemplary aspects of the following detailed description of the present disclosure and do not limit the scope of the present disclosure.

[0019] Beneficial Effects

[0020] According to the present disclosure, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency can be provided.

[0021] In addition, according to the present disclosure, an image encoding / decoding method and apparatus capable of improving encoding / decoding efficiency by performing adaptive color transform can be provided.

[0022] In addition, according to the present disclosure, a method of transmitting a bit stream generated by the image encoding method or apparatus according to the present disclosure may be provided.

[0023] In addition, according to the present disclosure, a recording medium storing a bit stream generated by the image encoding method or apparatus according to the present disclosure can be provided.

[0024] In addition, according to the present disclosure, a recording medium storing a bit stream received and decoded by the image decoding device according to the present disclosure and used to reconstruct an image can be provided.

[0025] Those skilled in the art will understand that the effects that can be achieved by the present disclosure are not limited to what has been specifically described above, and other advantages of the present disclosure will be more clearly understood from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a diagram schematically showing a video encoding system to which an embodiment of the present disclosure is applicable.

[0027] Figure 2 is a view schematically showing an image encoding device to which an embodiment of the present disclosure is applicable.

[0028] Figure 3 is a view schematically showing an image decoding device to which an embodiment of the present disclosure is applicable.

[0029] Figure 4 is a view showing a segmentation structure of an image according to an embodiment.

[0030] Figure 5 is a view showing an embodiment of a partition type of a block according to a multi-type tree structure.

[0031] Figure 6 is a diagram illustrating a signaling mechanism of block partitioning information in a structure of a quadtree with nested multi-type trees according to the present disclosure.

[0032] Figure 7 is a diagram showing an embodiment of dividing a CTU into a plurality of CUs.

[0033] Figure 8 is a diagram showing neighboring reference samples according to an embodiment.

[0034] Fig. 9 and Fig.10 is a view showing intra prediction according to an embodiment.

[0035] Fig.11 It is a diagram showing an embodiment of a decoding process to which ACT is applied.

[0036] Fig.12 is a diagram showing an embodiment of a sequence parameter set syntax table for signaling syntax elements related to an ACT.

[0037] Figures 13 to 19 is a diagram successively showing an embodiment of a syntax table for signaling a coding basis of syntax elements related to an ACT.

[0038] Fig. 20 is a view showing a coding tree syntax according to an embodiment.

[0039] Fig.21 is a view showing a block diagram of CABAC according to an embodiment for encoding one syntax element.

[0040] Figure 22 to Figure 25 is a view showing entropy encoding and decoding according to an embodiment.

[0041] Fig.26 and Fig. 27 is a view showing an example of a picture decoding and encoding process according to an embodiment.

[0042] Fig.28 is a view showing a hierarchical structure of a coded image according to an embodiment.

[0043] Fig.29 is a view illustrating a method of encoding residual samples of BDPCM according to an embodiment.

[0044] Fig.30 is a view showing a modified quantized residual block generated by performing BDPCM according to an embodiment.

[0045] Fig.31 is a flowchart illustrating a process of encoding a current block by applying BDPCM in an image encoding apparatus according to an embodiment.

[0046] Fig.32 is a flowchart illustrating a process of reconstructing a current block by applying BDPCM in an image decoding apparatus according to an embodiment.

[0047] Figure 33 to Figure 35 is a view schematically showing a syntax for signaling information about BDPCM.

[0048] Figure 36 to Figure 51 is a diagram showing a syntax table for signaling an ACT syntax element according to various individual embodiments of the present disclosure.

[0049] Fig.52 is a view showing an image decoding method according to an embodiment.

[0050] Fig.53 is a view showing an image encoding method according to an embodiment.

[0051] Fig.54 is a diagram showing a content streaming system to which an embodiment of the present disclosure is applicable. DETAILED DESCRIPTION

[0052] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to facilitate implementation by those skilled in the art. However, the present disclosure may be implemented in various forms and is not limited to the embodiments described herein.

[0053] When describing the present disclosure, if it is determined that the detailed description of related known functions or configurations makes the scope of the present disclosure unnecessarily ambiguous, the detailed description thereof will be omitted. In the drawings, parts irrelevant to the description of the present disclosure are omitted, and like reference numerals are given to like parts.

[0054] In the present disclosure, when a component is "connected", "coupled" or "linked" to another component, it may include not only a direct connection relationship but also an indirect connection relationship with intermediate components. In addition, when a component "includes" or "has" other components, unless otherwise specified, it means that other components may also be included, rather than excluding other components.

[0055] In the present disclosure, the terms first, second, etc. may only be used to distinguish one component from other components, and do not limit the order or importance of the components unless otherwise specified. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment may be referred to as a first component in another embodiment.

[0056] In the present disclosure, components that are distinguished from each other are intended to clearly describe each feature and do not mean that the components must be separated. That is, multiple components can be integrated or implemented in one hardware or software unit, or one component can be distributed and implemented in multiple hardware or software units. Therefore, even if not specifically stated, such implementation of component integration or component distribution is also included in the scope of the present disclosure.

[0057] In the present disclosure, the components described in various embodiments are not necessarily indispensable components, and some components may be optional components. Therefore, the embodiments consisting of a subset of the components described in the embodiments are also included in the scope of the present disclosure. In addition, the embodiments including other components in addition to the components described in various embodiments are included in the scope of the present disclosure.

[0058] The present disclosure relates to encoding and decoding of images. Unless otherwise defined in the present disclosure, terms used in the present disclosure may have general meanings commonly used in the technical field to which the present disclosure belongs.

[0059] In the present disclosure, "video" may mean a collection of a series of images according to the passage of time. "Picture" generally means the basis for representing an image in a specific time period, and a slice / patch is a coding basis that constitutes a part of a picture in coding. A picture may be composed of one or more slices / patch. In addition, a slice / patch may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / patch. A picture may be composed of one or more patch groups. A patch group may include one or more patches. A tile may refer to a quadrilateral area of ​​a CTU row within a patch in a picture. A patch may include one or more tiles. A tile may refer to a quadrilateral area of ​​a CTU row within a patch. A patch may be divided into multiple tiles, each of which may include one or more CTU rows belonging to the patch. A patch that is not divided into multiple tiles may also be treated as a tile.

[0060] In the present disclosure, "pixel" or "picture element" may refer to the smallest unit constituting a picture (or image). In addition, "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, or may represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component.

[0061] In the present disclosure, a "unit" may refer to a basic unit of image processing. A unit may include at least one of a specific area of ​​a picture and information related to the area. A unit may include a luminance block and two chrominance (e.g., Cb, Cr) blocks. In some cases, a unit may be used interchangeably with terms such as "sample array", "block" or "area". In general, an M×N block may include M columns and N rows of samples (or sample arrays) or a set (or array) of transform coefficients.

[0062] In the present disclosure, "current block" may mean one of "current coding block", "current coding unit", "coding target block", "decoding target block" or "processing target block". When prediction is performed, "current block" may mean "current prediction block" or "prediction target block". When transform (inverse transform) / quantization (dequantization) is performed, "current block" may mean "current transform block" or "transform target block". When filtering is performed, "current block" may mean "filtering target block".

[0063] In addition, in the present disclosure, unless explicitly stated as a chroma block, "current block" may mean "luminance block of the current block." "Chroma block of the current block" may be expressed by including an explicit description of a chroma block such as "chroma block" or "current chroma block."

[0064] In the present disclosure, the term " / " or "," may be interpreted as indicating "and / or". For example, the expressions "A / B" and "A, B" may mean "A and / or B". In addition, "A / B / C" and "A, B, C" may mean "at least one of A, B and / or C".

[0065] In the present disclosure, the term "or" should be interpreted as indicating "and / or". For example, the expression "A or B" may include 1) only "A", 2) only "B", and / or 3) both "A and B". In other words, in the present disclosure, the term "or" should be interpreted as indicating "additionally or alternatively".

[0066] Overview of Video Coding Systems

[0067] Figure 1 is a diagram illustrating a video encoding system according to the present disclosure.

[0068] The video encoding system according to the embodiment may include a source device 10 and a receiving device 20. The source device 10 may deliver encoded video and / or image information or data to the receiving device 20 via a digital storage medium or a network in the form of a file or a stream.

[0069] The source device 10 according to the embodiment may include a video source generator 11, an encoding device 12, and a transmitter 13. The receiving device 20 according to the embodiment may include a receiver 21, a decoding device 22, and a renderer 23. The encoding device 12 may be referred to as a video / image encoding device, and the decoding device 22 may be referred to as a video / image decoding device. The transmitter 13 may be included in the encoding device 12. The receiver 21 may be included in the decoding device 22. The renderer 23 may include a display and the display may be configured as a separate device or an external component.

[0070] The video source generator 11 can obtain the video / image by the process of capturing, synthesizing or generating the video / image. The video source generator 11 may include a video / image capturing device and / or a video / image generating device. The video / image capturing device may include, for example, one or more cameras, a video / image archive including previously captured videos / images, etc. The video / image generating device may include, for example, a computer, a tablet computer, and a smart phone, and may generate the video / image (electronically). For example, a virtual video / image may be generated by a computer, etc. In this case, the video / image capturing process may be replaced by a process of generating relevant data.

[0071] The encoding device 12 may encode the input video / image. For compression and encoding efficiency, the encoding device 12 may perform a series of processes such as prediction, transformation, and quantization. The encoding device 12 may output the encoded data (encoded video / image information) in the form of a bitstream.

[0072] The transmitter 13 transmits the encoded video / image information or data output in the form of a bit stream to the receiver 21 of the receiving device 20 in the form of a file or stream through a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter 13 may include an element for generating a media file in a predetermined file format and may include an element for transmitting through a broadcast / communication network. The receiver 21 may extract / receive a bit stream from a storage medium or a network and transmit the bit stream to a decoding device 22.

[0073] The decoding device 22 may decode a video / image by performing a series of processes corresponding to the operations of the encoding device 12, such as dequantization, inverse transformation, and prediction.

[0074] The renderer 23 may render the decoded video / image. The rendered video / image may be displayed through a display.

[0075] Overview of Image Coding Equipment

[0076] Figure 2 is a view schematically showing an image encoding device to which an embodiment of the present disclosure is applicable.

[0077] like Figure 2As shown, the image encoding device 100 may include an image segmenter 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-frame prediction unit 180, an intra-frame prediction unit 185, and an entropy encoder 190. The inter-frame prediction unit 180 and the intra-frame prediction unit 185 may be collectively referred to as a "prediction unit". The transformer 120, the quantizer 130, the dequantizer 140, and the inverse transformer 150 may be included in a residual processor. The residual processor may also include a subtractor 115.

[0078] In some embodiments, all or at least some of the components configuring the image encoding apparatus 100 may be configured by one hardware component (eg, an encoder or a processor). In addition, the memory 170 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium.

[0079] The image segmenter 110 may segment the input image (or picture or frame) input to the image encoding device 100 into one or more processing units. For example, the processing unit may be referred to as a coding unit (CU). The coding unit may be obtained by recursively segmenting a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree binary tree ternary tree (QT / BT / TT) structure. For example, a coding unit may be segmented into a plurality of coding units of a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. For the segmentation of the coding unit, the quadtree structure may be applied first, and the binary tree structure and / or the ternary tree structure may be applied later. The encoding process according to the present disclosure may be performed based on the final coding unit that is no longer segmented. The maximum coding unit may be used as the final coding unit, or the coding unit of a deeper depth obtained by segmenting the maximum coding unit may be used as the final coding unit. Here, the encoding process may include the prediction, transformation, and reconstruction processes described later. As another example, the processing unit of the encoding process may be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transform unit may be divided or partitioned from the final coding unit. The prediction unit may be a sample prediction unit, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient.

[0080] The prediction unit (inter prediction unit 180 or intra prediction unit 185) may perform prediction on the block to be processed (current block) and generate a prediction block including prediction samples of the current block. The prediction unit may determine whether to apply intra prediction or inter prediction based on the current block or CU. The prediction unit may generate various information related to the prediction of the current block and transmit the generated information to the entropy encoder 190. The information about the prediction may be encoded in the entropy encoder 190 and output in the form of a bitstream.

[0081] The intra prediction unit 185 can predict the current block by referring to the samples in the current picture. Depending on the intra prediction mode and / or the intra prediction technology, the reference samples can be located in the neighbors of the current block or can be placed separately. The intra prediction mode may include multiple non-directional modes and multiple directional modes. The non-directional mode may include, for example, a DC mode and a plane mode. Depending on the level of detail of the prediction direction, the directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes. However, this is only an example, and more or fewer directional prediction modes may be used according to the settings. The intra prediction unit 185 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0082] The inter prediction unit 180 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc. The reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter prediction unit 180 may configure a motion information candidate list based on the neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter prediction unit 180 may use the motion information of the neighboring block as the motion information of the current block. In the case of skip mode, unlike the merge mode, the residual signal may not be transmitted. In the case of motion vector prediction (MVP) mode, the motion vector of the neighboring block may be used as a motion vector predictor, and the motion vector of the current block may be signaled by encoding the motion vector difference and an indicator of the motion vector predictor. The motion vector difference may mean the difference between the motion vector of the current block and the motion vector predictor.

[0083] The prediction unit may generate a prediction signal based on various prediction methods and prediction techniques described below. For example, the prediction unit may not only apply intra prediction or inter prediction, but may also apply intra prediction and inter prediction simultaneously to predict the current block. The prediction method of simultaneously applying intra prediction and inter prediction to predict the current block may be referred to as combined inter and intra prediction (CIIP). In addition, the prediction unit may perform intra block copying (IBC) to predict the current block. Intra block copying may be used for content image / video encoding of games, etc., such as screen content coding (SCC). IBC is a method of predicting the current picture using a previously reconstructed reference block in the current picture at a position separated by a predetermined distance from the current block. When IBC is applied, the position of the reference block in the current picture may be encoded as a vector (block vector) corresponding to a predetermined distance. IBC basically performs prediction in the current picture, but may be performed similarly to inter prediction because the reference block is derived within the current picture. That is, IBC may use at least one of the inter prediction techniques described in the present disclosure.

[0084] The prediction signal generated by the prediction unit can be used to generate a reconstruction signal or to generate a residual signal. The subtractor 115 can generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output from the prediction unit from the input image signal (original block or original sample array). The generated residual signal can be transmitted to the transformer 120.

[0085] The transformer 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a karhunen-loève transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, GBT refers to a transform obtained from a graph when relationship information between pixels is represented by a graph. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process may be applied to square pixel blocks of the same size or may be applied to blocks of variable size rather than square.

[0086] The quantizer 130 may quantize the transform coefficients and transmit them to the entropy encoder 190. The entropy encoder 190 may encode the quantized signal (information about the quantized transform coefficients) and output a bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantizer 130 may rearrange the quantized transform coefficients in block form into a one-dimensional vector form based on a coefficient scanning order, and generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0087] The entropy encoder 190 may perform various encoding methods, such as, for example, exponential Golomb, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), etc. The entropy encoder 190 may encode information required for video / image reconstruction (e.g., values ​​of syntax elements, etc.) other than quantized transform coefficients together or separately. The encoded information (e.g., encoded video / image information) may be transmitted or stored in units of a network abstraction layer (NAL) in the form of a bitstream. The video / image information may also include information about various parameter sets, such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. The signaled information, the transmitted information, and / or the syntax elements described in the present disclosure may be encoded and included in the bitstream through the above encoding process.

[0088] The bit stream may be transmitted through a network or may be stored in a digital storage medium. The network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) that transmits a signal output from the entropy encoder 190 and / or a storage unit (not shown) that stores the signal may be included as an internal / external element of the image encoding device 100. Alternatively, a transmitter may be provided as a component of the entropy encoder 190.

[0089] The quantized transform coefficients output from the quantizer 130 may be used to generate a residual signal. For example, the residual signal (residual block or residual sample) may be reconstructed by applying dequantization and inverse transformation to the quantized transform coefficients via the dequantizer 140 and the inverse transformer 150.

[0090] The adder 155 adds the reconstructed residual signal to the prediction signal output from the inter-frame prediction unit 180 or the intra-frame prediction unit 185 to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). If the block to be processed has no residual (such as the case where the skip mode is applied), the prediction block can be used as a reconstructed block. The adder 155 can be called a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture, and can be used for inter-frame prediction of the next picture by filtering as described below.

[0091] The filter 160 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 160 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 170, specifically, in the DPB of the memory 170. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc. The filter 160 can generate various information related to filtering and transmit the generated information to the entropy encoder 190, as described later in the description of each filtering method. The information related to filtering can be encoded by the entropy encoder 190 and output in the form of a bit stream.

[0092] The modified reconstructed picture transferred to the memory 170 may be used as a reference picture in the inter prediction unit 180. When inter prediction is applied by the image encoding apparatus 100, prediction mismatch between the image encoding apparatus 100 and the image decoding apparatus may be avoided and encoding efficiency may be improved.

[0093] The DPB of the memory 170 may store the modified reconstructed picture for use as a reference picture in the inter-frame prediction unit 180. The memory 170 may store the motion information of the block for deriving (or encoding) the motion information in the current picture and / or the motion information of the reconstructed block in the picture. The stored motion information may be transmitted to the inter-frame prediction unit 180 and used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 170 may store the reconstructed samples of the reconstructed blocks in the current picture and may transmit the reconstructed samples to the intra-frame prediction unit 185.

[0094] Overview of Image Decoding Equipment

[0095] Figure 3 is a view schematically showing an image decoding device to which an embodiment of the present disclosure is applicable.

[0096] like Figure 3 As shown, the image decoding device 200 may include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame prediction unit 260, and an intra-frame prediction unit 265. The inter-frame prediction unit 260 and the intra-frame prediction unit 265 may be collectively referred to as a "prediction unit". The dequantizer 220 and the inverse transformer 230 may be included in a residual processor.

[0097] According to an embodiment, all or at least some of the plurality of components configuring the image decoding apparatus 200 may be configured by hardware components (eg, a decoder or a processor). In addition, the memory 250 may include a decoded picture buffer (DPB) or may be configured by a digital storage medium.

[0098] The image decoding device 200 having received a bit stream including video / image information may perform the same Figure 2 The image encoding device 100 may reconstruct the image by processing corresponding to the processing performed by the image encoding device 100. For example, the image decoding device 200 may perform decoding using a processing unit applied in the image encoding device. Therefore, the processing unit of decoding may be, for example, a coding unit. The coding unit may be obtained by partitioning the coding tree unit or the maximum coding unit. The reconstructed image signal decoded and output by the image decoding device 200 may be reproduced by a reproduction device (not shown).

[0099] The image decoding device 200 may receive the image in the form of a bit stream from Figure 2The received signal can be decoded by the entropy decoder 210. For example, the entropy decoder 210 can parse the bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. The image decoding device may also decode the picture based on the information about the parameter set and / or the general constraint information. The information and / or syntax elements signaled / received described in the present disclosure may be decoded and obtained from the bitstream through a decoding process. For example, the entropy decoder 210 decodes the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and outputs the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients of the residual. More specifically, the CABAC entropy decoding method can receive a bin corresponding to each syntax element in the bitstream, use the decoding target syntax element information, the decoding information of the neighboring block and the decoding target block, or the information of the symbol / bin decoded in the previous stage to determine the context model, perform arithmetic decoding on the bin by predicting the probability of occurrence of the bin according to the determined context model, and generate a symbol corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method can update the context model by using the information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model. The information related to prediction among the information decoded by the entropy decoder 210 can be provided to the prediction unit (inter-frame prediction unit 260 and intra-frame prediction unit 265), and the residual value (ie, quantized transform coefficient and related parameter information) entropy decoded in the entropy decoder 210 can be input to the dequantizer 220. In addition, the information about filtering among the information decoded by the entropy decoder 210 can be provided to the filter 240. In addition, a receiver (not shown) for receiving a signal output from the image encoding apparatus may be further configured as an internal / external element of the image decoding apparatus 200 , or the receiver may be a component of the entropy decoder 210 .

[0100] In addition, the image decoding device according to the present disclosure may be referred to as a video / image / picture decoding device. The image decoding device may be classified into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 210. The sample decoder may include a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame prediction unit 260, or at least one of an intra-frame prediction unit 265.

[0101] The dequantizer 220 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 may rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement may be performed based on the coefficient scanning order performed in the image encoding device. The dequantizer 220 may dequantize the quantized transform coefficients by using a quantization parameter (e.g., quantization step size information) and obtain the transform coefficients.

[0102] The inverse transformer 230 may inversely transform the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0103] The prediction unit may perform prediction on the current block and generate a prediction block including prediction samples of the current block. The prediction unit may determine whether to apply intra prediction or inter prediction to the current block based on the information about prediction output from the entropy decoder 210, and may determine a specific intra / inter prediction mode (prediction technique).

[0104] The same as described in the prediction unit of the image encoding device 100 , the prediction unit can generate a prediction signal based on various prediction methods (techniques) described later.

[0105] The intra prediction unit 265 may predict the current block by referring to samples in the current picture. The description of the intra prediction unit 185 is also applicable to the intra prediction unit 265.

[0106] The inter-frame prediction unit 260 can derive the prediction block of the current block based on the reference block (reference sample array) specified by the motion vector on the reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter-frame prediction mode, the motion information can be predicted in units of blocks, sub-blocks or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter-frame prediction unit 260 may configure a motion information candidate list based on the neighboring blocks, and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes, and information about the prediction may include information indicating the inter-frame prediction mode of the current block.

[0107] The adder 235 can generate a reconstruction signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the prediction unit (including the inter-frame prediction unit 260 and / or the intra-prediction unit 265). If the block to be processed has no residual (such as in the case of applying the skip mode), the prediction block can be used as the reconstructed block. The description of the adder 155 is also applicable to the adder 235. The adder 235 can be called a reconstructor or a reconstructed block generator. The generated reconstruction signal can be used for intra-frame prediction of the next block to be processed in the current picture, and can be used for inter-frame prediction of the next picture by filtering as described below.

[0108] The filter 240 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 240 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 250, specifically, in the DPB of the memory 250. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc.

[0109] The (modified) reconstructed picture stored in the DPB of the memory 250 can be used as a reference picture in the inter-frame prediction unit 260. The memory 250 can store the motion information of the block for deriving (or decoding) the motion information in the current picture and / or the motion information of the reconstructed block in the picture. The stored motion information can be transmitted to the inter-frame prediction unit 260 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 250 can store the reconstructed samples of the reconstructed block in the current picture and transmit the reconstructed samples to the intra-frame prediction unit 265.

[0110] In the present disclosure, the implementations described in the filter 160, the inter-frame prediction unit 180 and the intra-frame prediction unit 185 of the image encoding device 100 can be equally or correspondingly applied to the filter 240, the inter-frame prediction unit 260 and the intra-frame prediction unit 265 of the image decoding device 200.

[0111] Overview of Image Segmentation

[0112] The video / image encoding method according to the present disclosure can be performed based on the following image segmentation structure. Specifically, the prediction, residual processing ((inverse) transform, (de)quantization, etc.), syntax element encoding and filtering processes described later can be performed based on the CTU, CU (and / or TU, PU) derived from the image segmentation structure. The image can be segmented in block units and the block segmentation process can be performed in the image segmentor 110 of the encoding device. The segmentation related information can be encoded by the entropy encoder 190 and sent to the image decoding device in the form of a bit stream. The entropy decoder 210 of the image decoding device can derive the block segmentation structure of the current picture based on the segmentation related information obtained from the bit stream, and based on this, a series of processes (e.g., prediction, residual processing, block / picture reconstruction, in-loop filtering, etc.) can be performed to perform image decoding.

[0113] A picture may be partitioned into a sequence of coding tree units (CTUs). Figure 4 An example of a picture being partitioned into CTUs is shown. A CTU may correspond to a coding tree block (CTB). Alternatively, a CTU may include a coding tree block of luma samples and two corresponding coding tree blocks of chroma samples. For example, for a picture containing three sample arrays, a CTU may include one N×N block of luma samples and two corresponding blocks of chroma samples.

[0114] Overview of CTU Segmentation

[0115] As described above, a coding unit may be obtained by recursively partitioning a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree / binary tree / ternary tree (QT / BT / TT) structure. For example, a CTU may be first partitioned into a quadtree structure. Thereafter, the leaf nodes of the quadtree structure may be further partitioned by a multi-type tree structure.

[0116] Partitioning according to the quadtree means that the current CU (or CTU) is equally divided into four. By partitioning according to the quadtree, the current CU can be partitioned into four CUs with the same width and the same height. When the current CU is no longer partitioned into a quadtree structure, the current CU corresponds to a leaf node of the quadtree structure. The CU corresponding to the leaf node of the quadtree structure may no longer be partitioned and may be used as the above-mentioned final coding unit. Alternatively, the CU corresponding to the leaf node of the quadtree structure may be further partitioned by a multi-type tree structure.

[0117] Figure 5 2 is a view showing a partition type of a block according to a multi-type tree structure. The partition according to the multi-type tree structure may include two types of partitions according to a binary tree structure and two types of partitions according to a ternary tree structure.

[0118] The two types of splits according to the binary tree structure may include vertical binary split (SPLIT_BT_VER) and horizontal binary split (SPLIT_BT_HOR). Vertical binary split (SPLIT_BT_VER) means that the current CU is equally split into two in the vertical direction. Figure 4 As shown in FIG, through vertical binary splitting, two CUs with the same height as the current CU and half the width of the current CU can be generated. Horizontal binary splitting (SPLIT_BT_HOR) means that the current CU is equally divided into two in the horizontal direction. Figure 5 As shown, through horizontal binary partitioning, two CUs with a height half of the height of the current CU and the same width as the current CU can be generated.

[0119] The two types of splits according to the ternary tree structure may include vertical ternary split (SPLIT_TT_VER) and horizontal ternary split (SPLIT_TT_HOR). In vertical ternary split (SPLIT_TT_VER), the current CU is split in a vertical direction at a ratio of 1:2:1. Figure 5 As shown, through vertical trisection, two CUs with the same height as the current CU and a width of 1 / 4 of the current CU width and one CU with the same height as the current CU and a width of half the current CU width can be generated. In horizontal trisection (SPLIT_TT_HOR), the current CU is split in the horizontal direction at a ratio of 1:2:1. Figure 4 As shown, through horizontal trifurcated division, two CUs whose height is 1 / 4 of the height of the current CU and whose width is the same as the current CU and one CU whose height is half of the height of the current CU and whose width is the same as the current CU can be generated.

[0120] Figure 6 is a diagram illustrating a signaling mechanism of block partitioning information in a structure of a quadtree with nested multi-type trees according to the present disclosure.

[0121] Here, the CTU is regarded as the root node of the quadtree and is split into a quadtree structure for the first time. Information (e.g., qt_split_flag) indicating whether quadtree partitioning is performed on the current CU (CTU or node (QT_node) of the quadtree) is notified by a signal. For example, when qt_split_flag has a first value (e.g., "1"), the current CU can be split by the quadtree. In addition, when qt_split_flag has a second value (e.g., "0"), the current CU is not split by the quadtree, but becomes a leaf node (QT_leaf_node) of the quadtree. Each quadtree leaf node can then be further split into a multi-type tree structure. That is, the leaf node of the quadtree can become a node (MTT_node) of a multi-type tree. In the multi-type tree structure, a first flag (e.g., Mtt_split_cu_flag) is notified by a signal to indicate whether the current node is additionally split. If the corresponding node is additionally split (for example, if the first flag is 1), the second flag (for example, Mtt_split_cu_vertical_flag) may be signaled to indicate the split direction. For example, the split direction may be a vertical direction when the second flag is 1, and a horizontal direction when the second flag is 0. Then, a third flag (for example, Mtt_split_cu_binary_flag) may be signaled to indicate whether the split type is a binary split type or a ternary split type. For example, the split type may be a binary split type when the third flag is 1, and a ternary split type when the third flag is 0. The nodes of the multi-type tree obtained by binary splitting or ternary splitting may be further split into a multi-type tree structure. However, the nodes of the multi-type tree may not be split into a quadtree structure. If the first flag is 0, the corresponding node of the multi-type tree is no longer split, but becomes a leaf node (MTT_leaf_node) of the multi-type tree. The CU corresponding to the leaf node of the multi-type tree may be used as the above-mentioned final coding unit.

[0122] Based on mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, a multi-type tree partitioning mode (MttSplitMode) of a CU may be derived as shown in the following Table 1. In the following description, a multi-tree partitioning mode may be referred to as a multi-tree partitioning type or simply a partitioning type.

[0123] [Table 1]

[0124] MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1

[0125] Figure 7 is a view showing an example of splitting a CTU into a plurality of CUs by applying a multi-type tree after applying a quadtree. Figure 7 , the bold block edge 710 represents quadtree partitioning, while the remaining edge 720 represents multi-type tree partitioning. The CU may correspond to a coding block (CB). In an embodiment, the CU may include one coding block of luma samples and two coding blocks of chroma samples corresponding to the luma samples. The chroma component (sample) CB or TB size may be derived based on the luma component (sample) CB or TB size based on the component ratio according to the color format (chroma format, e.g., 4:4:4, 4:2:2, 4:2:0, etc.) of the picture / image. In the case of a 4:4:4 color format, the chroma component CB / TB size may be set equal to the luma component CB / TB size. In the case of a 4:2:2 color format, the width of the chroma component CB / TB may be set to half the width of the luma component CB / TB and the height of the chroma component CB / TB may be set to the height of the luma component CB / TB. In the case of a 4:2:0 color format, the width of the chroma component CB / TB may be set to half the width of the luma component CB / TB and the height of the chroma component CB / TB may be set to half the height of the luma component CB / TB.

[0126] In an embodiment, when the size of the CTU is 128 based on the luma sample unit, the size of the CU may have a size from 128×128 to 4×4, which is the same size as the CTU. In an embodiment, in the case of a 4:2:0 color format (or chroma format), the chroma CB size may have a size from 64×64 to 2×2.

[0127] Furthermore, in an embodiment, the CU size and the TU size may be the same. Alternatively, there may be multiple TUs in a CU region. The TU size generally indicates the luma component (sample) transform block (TB) size.

[0128] The TU size can be derived based on the maximum allowed TB size maxTbSize as a predetermined value. For example, when the CU size is larger than maxTbSize, multiple TUs (TBs) with maxTbSize can be derived from the CU, and transform / inverse transform can be performed in units of TU (TB). For example, the maximum allowed luma TB size may be 64×64 and the maximum allowed chroma TB size may be 32×32. If the width or height of the CB split according to the tree structure is larger than the maximum transform width or height, the CB may be automatically (or implicitly) split until the TB size limits in the horizontal and vertical directions are met.

[0129] In addition, for example, when intra prediction is applied, the intra prediction mode / type may be derived in units of CU (or CB), and the neighboring reference sample derivation and prediction sample generation process may be performed in units of TU (or TB). In this case, there may be one or more TUs (or TBs) in a CU (or CB) region, and in this case, multiple TUs or (TBs) may share the same intra prediction mode / type.

[0130] In addition, for a quadtree coding tree scheme with nested multi-type trees, the following parameters may be signaled from an image encoding device to a decoding device as SPS syntax elements. For example, at least one of a CTU size as a parameter representing the root node size of a quadtree, a MinQTSize as a parameter representing the minimum allowed quadtree leaf node size, a MaxBtSize as a parameter representing the maximum allowed binary tree root node size, a MaxTtSize as a parameter representing the maximum allowed ternary tree root node size, a MaxMttDepth as a parameter representing the maximum allowed hierarchical depth of multi-type tree partitioning from a quadtree leaf node, a MinBtSize as a parameter representing the minimum allowed binary tree leaf node size, or a MinTtSize as a parameter representing the minimum allowed ternary tree leaf node size may be signaled.

[0131] As an embodiment using a 4:2:0 chroma format, the CTU size may be set to a 128×128 luminance block and two 64×64 chroma blocks corresponding to the luminance block. In this case, MinOTSize may be set to 16×16, MaxBtSize may be set to 128×128, MaxTtSzie may be set to 64×64, MinBtSize and MinTtSize may be set to 4×4, and MaxMttDepth may be set to 4. Quadtree partitioning may be applied to the CTU to generate a quadtree leaf node. The quadtree leaf node may be referred to as a leaf QT node. The size of the quadtree leaf node may be from 16×16 size (e.g., MinOTSize) to 128×128 size (e.g., CTU size). If the leaf QT node is 128×128, it may not be additionally partitioned into a binary tree / ternary tree. This is because, in this case, even if it is partitioned, it exceeds MaxBtsize and MaxTtszie (e.g., 64×64). In other cases, the leaf QT node may be further split into a multi-type tree. Therefore, the leaf QT node is the root node of the multi-type tree, and the leaf QT node may have a multi-type tree depth (mttDepth) value of 0. If the multi-type tree depth reaches MaxMttdepth (for example, 4), further splitting may not be considered. If the width of the multi-type tree node is equal to MinBtSize and is less than or equal to 2×MinTtSize, further horizontal splitting may not be considered. If the height of the multi-type tree node is equal to MinBtSize and is less than or equal to 2×MinTtSize, further vertical splitting may not be considered. When splitting is not considered, the image encoding device may skip the signaling of the splitting information. In this case, the image decoding device may derive splitting information having a predetermined value.

[0132] In addition, one CTU may include a coding block of luma samples (hereinafter referred to as "luminance block") and two coding blocks of chroma samples corresponding thereto (hereinafter referred to as "chroma blocks"). The above coding tree scheme may be applied equally or individually to the luma block and chroma block of the current CU. Specifically, the luma block and chroma block in one CTU may be partitioned into the same block tree structure, and in this case, the tree structure is represented as SINGLE_TREE. Alternatively, the luma block and chroma block in one CTU may be partitioned into separate block tree structures, and in this case, the tree structure may be represented as DUAL_TREE. That is, when the CTU is partitioned into dual trees, there may be a block tree structure for the luma block and a block tree structure for the chroma block, respectively. In this case, the block tree structure for the luma block may be referred to as DUAL_TREE_LUMA, and the block tree structure for the chroma component may be referred to as DUAL_TREE_CHROMA. For P and B slices / block groups, the luma block and the chroma block in one CTU may be restricted to have the same coding tree structure. However, for I slices / patch groups, luma blocks and chroma blocks may have separate block tree structures from each other. If a separate block tree structure is applied, luma CTBs may be partitioned into CUs based on a specific coding tree structure, and chroma CTBs may be partitioned into chroma CUs based on another coding tree structure. That is, this means that a CU in an I slice / patch group to which a separate block tree structure is applied may include a coding block of a luma component or a coding block of two chroma components, and a CU of a P or B slice / patch group may include blocks of three color components (one luma component and two chroma components).

[0133] Although a quadtree coding tree structure with nested multi-type trees has been described, the structure for partitioning the CU is not limited thereto. For example, the BT structure and the TT structure may be interpreted as concepts included in a multi-partition tree (MPT) structure, and the CU may be interpreted as being partitioned by the QT structure and the MPT structure. In an example where the CU is partitioned by the QT structure and the MPT structure, a syntax element (e.g., MPT_split_type) including information about how many blocks a leaf node of the QT structure is partitioned into and a syntax element (e.g., MPT_split_mode) including information about which of the vertical and horizontal directions a leaf node of the QT structure is partitioned into may be signaled to determine the partition structure.

[0134] In another example, the CU may be partitioned in a manner different from the QT structure, the BT structure, or the TT structure. That is, unlike partitioning a CU of a lower depth into 1 / 4 of a CU of a higher depth according to the QT structure, partitioning a CU of a lower depth into 1 / 2 of a CU of a higher depth according to the BT structure, or partitioning a CU of a lower depth into 1 / 4 or 1 / 2 of a CU of a higher depth according to the TT structure, in some cases, a CU of a lower depth may be partitioned into 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 of a CU of a higher depth, and the method of partitioning a CU is not limited thereto.

[0135] The quadtree coding block structure with multi-type trees can provide a very flexible block segmentation structure. Due to the segmentation types supported in the multi-type tree, different segmentation patterns can potentially produce the same coding block structure in some cases. In the image encoding device and decoding device, by limiting the occurrence of such redundant segmentation patterns, the amount of data of the segmentation information can be reduced.

[0136] In addition, in the encoding and decoding of the video / image according to the present document, the image processing basis may have a hierarchical structure. A picture may be divided into one or more tiles, tiles, slices and / or tile groups. A slice may include one or more tiles. A tile may include one or more CTU rows within a tile. A slice may include tiles of a picture, wherein the number of tiles is an integer. A tile group may include one or more tiles. A tile may include one or more CTUs. A CTU may be divided into one or more CUs. A tile may be a quadrilateral area consisting of a specific tile row and a specific tile column including multiple CTUs within a picture. A tile group may include a tile raster scanned according to a tile within a picture, wherein the number of tiles is an integer. A slice header may carry information / parameters applicable to a corresponding slice (a block in a slice). When an encoding device or a decoding device has a multi-core processor, the encoding / decoding process for tiles, slices, tiles and / or tile groups may be performed in parallel.

[0137] In the present disclosure, the names or concepts of slices or mosaic groups may be used interchangeably. That is, a mosaic group header may be referred to as a slice header. Herein, a slice may have one of slice types including an intra (I) slice, a predicted (P) slice, and a bi-predicted (B) slice. For blocks within an I slice, inter prediction is not used for prediction, and only intra prediction may be used. Even in this case, the original sample values ​​may be encoded and signaled without prediction. For blocks within a P slice, intra prediction or inter prediction may be used. When inter prediction is used, only single prediction may be used. In addition, for blocks within a B slice, intra prediction or inter prediction may be used. When inter prediction is used, bi-prediction as the maximum range may be used.

[0138] According to the characteristics of the video image (e.g., resolution) or considering coding efficiency or parallel processing, the encoding device can determine the size of the patch / patch group, tile, slice, and the maximum and minimum coding unit. In addition, information about this or information for deriving this can be included in the bitstream.

[0139] The decoding device can obtain information indicating whether a tile / tile group, tile, slice, or CTU within a tile of the current picture is divided into multiple coding units. The encoding device and the decoding device only signal such information under certain conditions, thereby improving encoding efficiency.

[0140] The slice header (slice header syntax) may include information / parameters that are commonly applicable to slices. APS (APS syntax) or PPS (PPS syntax) may include information / parameters that are commonly applicable to one or more pictures. SPS (SPS syntax) may include information / parameters that are commonly applicable to one or more sequences. VPS (VPS syntax) may include information / parameters that are commonly applicable to multiple layers. DPS (DPS syntax) may include information / parameters that are commonly applicable to the entire video. DPS may include information / parameters related to a combination of coded video sequences (CVS).

[0141] In addition, for example, information on the partitioning and configuration of tiles / tile groups / tiles / slices may be constructed through a high-level syntax during the encoding stage and sent to a decoding device in the form of a bitstream.

[0142] Overview of intra prediction

[0143] Hereinafter, intra prediction performed by the above encoding device and decoding device will be described in detail. Intra prediction may refer to prediction of generating a prediction sample of a current block based on a reference sample in a picture to which the current block belongs (hereinafter, current picture).

[0144] This will refer to Figure 8 Description. When intra prediction is applied to the current block 801, neighboring reference samples to be used for intra prediction of the current block 801 may be derived. The neighboring reference samples of the current block may include: a total of 2×nH samples including a sample 811 adjacent to the left boundary of the current block of size nW×nH and a sample 812 adjacent to the lower left; a total of 2×nW samples including a sample 821 adjacent to the upper boundary of the current block and a sample 822 adjacent to the upper right; and one sample 831 adjacent to the upper left of the current block. Alternatively, the neighboring reference samples of the current block may include multiple columns of upper neighboring samples and multiple columns of left neighboring samples.

[0145] In addition, the neighboring reference samples of the current block may include: a total of nH samples 841 adjacent to the right boundary of the current block of size nW×nH; a total of nW samples 851 adjacent to the lower boundary of the current block; and one sample 842 adjacent to the lower right of the current block.

[0146] However, some neighboring reference samples of the current block have not been decoded or may not be available. In this case, the decoding device can construct neighboring reference samples to be used for prediction by replacing unavailable samples with available samples. Alternatively, the neighboring reference samples to be used for prediction can be constructed by interpolation of available samples.

[0147] When deriving neighboring reference samples, (i) the prediction samples may be derived based on the average or interpolation of the neighboring reference samples of the current block, or (ii) the prediction samples may be derived based on reference samples that exist in a specific (prediction) direction relative to the prediction samples among the neighboring reference samples of the current block. The case of (i) may be referred to as a non-directional mode or a non-angle mode, and the case of (ii) may be referred to as a directional mode or an angle mode. In addition, the prediction samples may be generated by interpolating the second neighboring samples and the first neighboring samples that are located in the opposite direction of the prediction direction of the intra-frame prediction mode of the current block among the neighboring reference samples. The above case may be referred to as linear interpolation intra-frame prediction (LIP). In addition, the chrominance prediction samples may be generated based on the luminance samples using a linear model. This case may be referred to as an LM mode. In addition, the temporary prediction samples of the current block may be derived based on the filtered neighboring reference samples, and the prediction samples of the current block may be derived by weighted summing the temporary prediction samples with at least one reference sample derived according to the intra-frame prediction mode among the existing neighboring reference samples (i.e., the unfiltered neighboring reference samples). The above situation may be referred to as position-dependent intra prediction (PDPC). In addition, a reference sample row with the highest prediction accuracy may be selected from a plurality of adjacent reference sample rows of the current block to derive a prediction sample by using a reference sample located in the prediction direction in the corresponding row. At this time, intra prediction coding may be performed by indicating (signaling) the reference sample row used to the decoding device. The above situation may be referred to as multi-reference row (MRL) intra prediction or MRL-based intra prediction. In addition, the current block may be divided into vertical or horizontal sub-partitions to perform intra prediction based on the same intra prediction mode, and adjacent reference samples may be derived and used based on each sub-partition. That is, in this case, the intra prediction mode of the current block is equally applied to the sub-partition, and adjacent reference samples are derived and used based on each sub-partition, thereby increasing intra prediction performance in some cases. This prediction method may be referred to as intra sub-partition (ISP) or intra prediction based on ISP. These intra prediction methods may be referred to as intra prediction types, which are distinguished from intra prediction modes (e.g., DC mode, plane mode, and directional mode). The intra prediction type may be referred to as various terms such as an intra prediction scheme or an additional intra prediction mode. For example, the intra prediction type (or additional intra prediction mode) may include at least one selected from the group of LIP, PDPC, MRL, and ISP described above. A general intra prediction method other than a specific intra prediction type such as LIP, PDPC, MRL, and ISP may be referred to as a normal intra prediction type. A normal intra prediction type may refer to a case where a specific intra prediction type is not applied and prediction may be performed based on the above intra prediction mode. In addition, post filtering may be performed on the derived prediction sample when necessary.

[0148] Specifically, the intra prediction process may include an intra prediction mode / type determination step, a neighboring reference sample derivation step, and a prediction sample derivation step based on the intra prediction mode / type. In addition, when necessary, a post-filtering step may be performed on the derived prediction samples.

[0149] In addition, in addition to the above-mentioned intra prediction types, affine linear weighted intra prediction (ALWIP) can also be used. ALWIP can be referred to as linear weighted intra prediction (LWIP) or matrix weighted intra prediction or matrix-based intra prediction (MIP). When MIP is applied to the current block, the prediction sample of the current block can be derived by: i) using neighboring reference samples that have undergone an averaging process, ii) performing a matrix vector multiplication process, and further performing iii) a horizontal / vertical interpolation process when necessary. The intra prediction mode for MIP may be different from the intra prediction mode used in LIP, PDPC, MRL, ISP intra prediction or normal intra prediction. The intra prediction mode for MIP may be referred to as MIP intra prediction mode, MIP prediction mode or MIP mode. For example, different matrices and offsets used in matrix vector multiplication can be set according to the intra prediction mode for MIP. Herein, the matrix may be referred to as a (MIP) weighting matrix, and the offset may be referred to as a (MIP) offset vector or a (MIP) offset vector. The detailed MIP method will be described later.

[0150] The block reconstruction process based on intra prediction and intra prediction unit in the encoding device may schematically include, for example, the following. Step S910 may be performed by the intra prediction unit 185 of the encoding device. Step S920 may be performed by a residual processor including at least one selected from the group consisting of the subtractor 115, the transformer 120, the quantizer 130, the dequantizer 140, and the inverse transformer 150 of the encoding device. Specifically, step S920 may be performed by the subtractor 115 of the encoding device. At step S930, prediction information may be derived by the intra prediction unit 185 and may be encoded by the entropy encoder 190. At step S930, residual information may be derived by the residual processor and may be encoded by the entropy encoder 190. Residual information is information about residual samples. The residual information may include information about quantized transform coefficients of the residual samples. As described above, the residual samples may be derived as transform coefficients by the transformer 120 of the encoding device, and the transform coefficients may be derived as quantized transform coefficients by the quantizer 130. Information about the quantized transform coefficients may be encoded by the entropy encoder 190 through a residual encoding process.

[0151] In step S910, the encoding device may perform intra prediction on the current block. The encoding device derives the intra prediction mode / type of the current block, derives the neighboring reference samples of the current block, and generates prediction samples in the current block based on the intra prediction mode / type and the neighboring reference samples. Herein, the process of determining the intra prediction mode / type, deriving the neighboring reference samples, and generating the prediction samples may be performed simultaneously, or any one process may be performed before the other processes. For example, although not shown, the intra prediction unit 185 of the encoding device may include an intra prediction mode / type determination unit, a reference sample derivation unit, and a prediction sample derivation unit. The intra prediction mode / type determination unit may determine the intra prediction mode / type of the current block, the reference sample derivation unit may derive the neighboring reference samples of the current block, and the prediction sample derivation unit may derive the prediction samples of the current block. In addition, when performing a prediction sample filtering process (to be described later), the intra prediction unit 185 may also include a prediction sample filter. The encoding device may determine a mode / type applied to the current block among multiple intra prediction modes / types. The encoding apparatus may compare RD costs of intra prediction modes / types and determine an optimal intra prediction mode / type for the current block.

[0152] In addition, the encoding device may perform a prediction sample filtering process. Prediction sample filtering may be referred to as post-filtering. Through the prediction sample filtering process, some or all prediction samples may be filtered. In some cases, the prediction sample filtering process may be omitted.

[0153] In step S920, the encoding device may generate residual samples of the current block based on the (filtered) prediction samples. The encoding device may compare the prediction samples in the original samples of the current block based on the phase, and may derive the residual samples.

[0154] In step S930, the encoding device may encode image information including information about intra prediction (prediction information) and residual information about residual samples. The prediction information may include intra prediction mode information and intra prediction type information. The encoding device may output the encoded image information in the form of a bit stream. The output bit stream may be transmitted to a decoding device via a storage medium or a network.

[0155] The residual information may include a residual encoding syntax (to be described later). The encoding device may derive a quantized transform coefficient by transforming / quantizing the residual samples. The residual information may include information about the quantized transform coefficient.

[0156] In addition, as described above, the encoding device can generate a reconstructed picture (including reconstructed samples and reconstructed blocks). To this end, the encoding device can perform dequantization / inverse transformation on the quantized transform coefficients and derive (modified) residual samples. The reason for performing dequantization / inverse transformation after transformation / quantization of the residual samples is to derive the same residual samples as the residual samples derived by the decoding device as described above. The encoding device can generate a reconstructed block including the reconstructed samples of the current block based on the predicted samples and the (modified) residual samples. Based on the reconstructed block, a reconstructed picture of the current picture can be generated. As described above, the in-loop filtering process can also be applied to the reconstructed picture.

[0157] The video / image decoding process based on intra prediction and intra prediction units in the decoding device may schematically include, for example, the following contents: The decoding device may perform operations corresponding to those performed by the encoding device.

[0158] Steps S1010 to S1030 may be performed by the intra prediction unit 265 of the decoding device. The prediction information in step S1010 and the residual information in step S1040 may be obtained from the bitstream by the entropy decoder 210 of the decoding device. A residual processor including a dequantizer 220 or an inverse transformer 230 or both of the decoding device may derive residual samples of the current block based on the residual information. Specifically, the dequantizer 220 of the residual processor may perform dequantization according to the quantized transform coefficient derived based on the residual information, and may derive the transform coefficient. The inverse transformer 230 of the residual processor may perform an inverse transform on the transform coefficient, and may derive the residual sample of the current block. Step S1050 may be performed by the adder 235 or the reconstructor of the decoding device.

[0159] Specifically, in step S1010, the decoding device may derive the intra prediction mode / type of the current block based on the received prediction information (intra prediction mode / type information). In step S1020, the decoding device may derive the neighboring reference samples of the current block. In step S1030, the decoding device may generate prediction samples in the current block based on the intra prediction mode / type and the neighboring reference samples. In this case, the decoding device may perform a prediction sample filtering process. Prediction sample filtering may be referred to as post filtering. Through the prediction sample filtering process, some or all prediction samples may be filtered. In some cases, the prediction sample filtering process may be omitted.

[0160] The decoding device may generate residual samples of the current block based on the received residual information. In step S1040, the decoding device may generate reconstructed samples of the current block based on the predicted samples and the residual samples and derive a reconstructed block including the reconstructed samples. Based on the reconstructed block, a reconstructed picture of the current picture may be generated. As described above, the in-loop filtering process may also be applied to the reconstructed picture.

[0161] Herein, although not shown, the intra prediction unit 265 of the decoding device may include an intra prediction mode / type determination unit, a reference sample derivation unit, and a prediction sample derivation unit. The intra prediction mode / type determination unit may determine the intra prediction mode / type of the current block based on the intra prediction mode / type information obtained from the entropy decoder 210. The reference sample derivation unit may derive the neighboring reference samples of the current block. The prediction sample derivation unit may derive the prediction sample of the current block. In addition, when performing the above-mentioned prediction sample filtering process, the intra prediction unit 265 may further include a prediction sample filter.

[0162] For example, the intra prediction mode information may include flag information (e.g., intra_luma_mpm_flag) indicating whether the most probable mode (MPM) or the remaining mode is applied to the current block. When MPM is applied to the current block, the prediction mode information may also include index information (e.g., intra_luma_mpm_idx) indicating one of the intra prediction mode candidates (MPM candidates). The intra prediction mode candidates (MPM candidates) may be constructed as an MPM candidate list or an MPM list. In addition, when MPM is not applied to the current block, the intra prediction mode information may also include remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra prediction modes other than the intra prediction mode candidates (MPM candidates). The decoding device may determine the intra prediction mode of the current block based on the intra prediction mode information. A separate MPM list may be constructed for the above-mentioned MIP.

[0163] In addition, the intra prediction type information may be implemented in various forms. For example, the intra prediction type information may include intra prediction type index information indicating one of the intra prediction types. As another example, the intra prediction type information may include reference sample row information indicating whether MRL is applied to the current block and which reference sample row is used when MRL is applied to the current block (e.g., intra_luma_ref_idx), ISP flag information indicating whether ISP is applied to the current block (e.g., intra_subpartitions_mode_flag), ISP type information indicating the partition type of the sub-partition when ISP is applied (e.g., intra_subpartitions_split_flag), flag information indicating whether PDCP is applied, or flag information indicating whether LIP is applied. In addition, the intra prediction type information may include a MIP flag indicating whether MIP is applied to the current block.

[0164] The intra-frame prediction mode information and / or the intra-frame prediction type information can be encoded / decoded by the encoding method described in this document. For example, the intra-frame prediction mode information and / or the intra-frame prediction type information can be encoded / decoded by entropy coding (e.g., CABAC, CAVLC) based on truncated (rice) binary code.

[0165] Overview of Adaptive Color Transform (ACT)

[0166] Adaptive Color Transform (ACT) is a color space transformation (conversion) technique for removing unnecessary overlap between color components and has been used in the HEVC Screen Content Extension. This can also be applied to VVC.

[0167] In HEVC Screen Content Extension (HEVC SCC extension), ACT has been used to adaptively transform the prediction residual from the existing color space to the YCgCo color space. One of the two color spaces can be optionally selected by signaling an ACT flag for each transform basis.

[0168] For example, a first value of the flag (eg, 1) may indicate that the residual of the transformed basis is encoded in the original color space. A second value of the flag (eg, 1) may indicate that the residual of the transformed basis is encoded in the YCgCo color space.

[0169] Fig.11 FIG. 4 is a diagram showing an embodiment of a decoding process to which ACT is applied. Fig.11 In an implementation manner, motion compensated prediction may correspond to inter-frame prediction in the present disclosure.

[0170] like Fig.11 As shown, the reconstructed picture (or reconstructed block, reconstructed sample array, reconstructed sample, reconstructed signal) can be generated based on the predicted output value and the residual output value. Herein, the residual output value can be an inverse transform output value. Herein, the inverse transform can be a normal inverse transform. Herein, the normal inverse transform can be an inverse transform based on MTS or an inverse low frequency non-separable transform (LFNST).

[0171] Herein, the prediction output value may be a prediction block, a prediction sample array, a prediction sample or a prediction signal. The residual output value may be a residual block, a residual sample array, a residual sample or a residual signal.

[0172] For example, in the case of an encoding device, an ACT process may be performed on a residual sample derived based on a prediction sample. In addition, an output value of the ACT process may be provided as an input of a normal transform process. Herein, the normal transform process may be an MTS-based transform or a LFNST.

[0173] The information (parameters) about the (inverse) ACT may be generated and encoded by an encoding device, and may be transmitted to a decoding device in the form of a bit stream.

[0174] The decoding device may obtain, parse, and decode (inverse) ACT related information (parameters), and may perform inverse ACT based on the (inverse) ACT related information (parameters).

[0175] Based on the inverse ACT, a (modified) residual sample (or residual block) can be derived. For example, the (transform) coefficient can be derived by applying dequantization to the quantized (transform) coefficient. In addition, the residual sample can be derived by performing an inverse transform on the (transform) coefficient. In addition, the (modified) residual sample can be obtained by applying the inverse ACT to the residual sample. The information (parameters) about the (inverse) ACT will be described in detail later.

[0176] In an embodiment, the core transform function used in HEVC may be used as the core transform function (transform kernel) of the color space transform. For example, matrices for forward transform and backward transform as shown in the following equations may be used.

[0177] [Formula 1]

[0178]

[0179] [Formula 2]

[0180]

[0181] Herein, C0, C1, and C2 may correspond to G, B, and R. Herein, G represents a green component, B represents a blue component, and R represents a red component. In addition, C0', C1', and C2' may correspond to Y, Cg, and Co. Herein, Y represents brightness, Cg represents green chrominance, and Co represents an orange chrominance component.

[0182] In addition, in order to compensate for the dynamic range change of the residual before and after the color transformation, QP adjustment can be applied to the transformed residual as (-5, -5, -3). The details of QP adjustment will be described later.

[0183] Furthermore, in the encoding and decoding processes according to the embodiments, when ACT is applicable, the following restrictions may be applied.

[0184] - In case of dual-tree encoding / decoding, ACT is disabled. For example, ACT can be applied only to single-tree encoding / decoding.

[0185] - ACT can be deactivated when ISP encoding and decoding is applied.

[0186] - ACT can be deactivated for chroma blocks with BDPCM applied. ACT can be activated only for luma blocks with BDPCM applied.

[0187] - CCLM can be deactivated when ACT can be applied.

[0188] Fig.12 is a diagram showing an embodiment of a sequence parameter set syntax table for signaling syntax elements related to an ACT.

[0189] Figures 13 to 19 is a diagram successively showing an embodiment of a syntax table for signaling a coding basis of syntax elements related to an ACT.

[0190] like Fig.12 As shown, as an ACT activation flag indicating whether ACT is activated in the decoding process, sps_act_enabled_flag 1210 can be used.

[0191] The first value (eg, 0) of sps_act_enabled_flag may indicate that ACT is not used, and the flag cu_act_enabled_flag 1310, 2910 indicating whether ACT is applied in the coding base is not provided in the syntax of the coding base.

[0192] A second value (eg, 1) of sps_act_enabled_flag may indicate that ACT may be used, and cu_act_enabled_flag may be provided in the syntax of the coding base.

[0193] When sps_act_enabled_flag is not obtained in a bitstream, the value of sps_act_enabled_flag may be derived as a first value (eg, 0).

[0194] In addition, if Fig.13 As shown, as an ACT flag indicating whether the residual of the current coding basis is encoded in the YCgCo color space, cu_act_enabled_flag 1310, 2710 can be used.

[0195] A first value of cu_act_enabled_flag (eg, 0) may indicate that the residual of the current coding basis is encoded in the original color space. A second value of cu_act_enabled_flag (eg, 1) may indicate that the residual of the current coding basis is encoded in the YCgCo color space.

[0196] When cu_act_enabled_flag is not provided in the bitstream, the flag may be derived as a first value (eg, 0). Herein, the original color space may be an RGB color space.

[0197] Transform-based QP derivation method using ACTQP offset

[0198] In an embodiment, in the scaling process for the transform coefficient, the derivation process of the quantization parameter and the Qp update process may be performed as follows: For example, the quantization parameter derivation process may be performed using the following parameters.

[0199] - Luma coordinates (xCb, yCb), which indicate the relative coordinates of the top left luma sample of the current coding block relative to the top left luma sample of the current picture

[0200] - The variable cbWidth indicates the width of the current coding block on a per luma sample basis

[0201] - The variable cbHeight, which indicates the height of the current coding block on a per luma sample basis

[0202] - variable treeType, which indicates whether a single tree (SINGLE_TREE) or a dual tree is used for partitioning the current coding tree node, and indicates whether the dual tree is a luma component dual tree (DAUL_TREE_LUMA) or a chroma component dual tree (DAUL_TREE_CHROMA) when a dual tree is used

[0203] In this process, the luma quantization parameter Qp'Y, and the chroma quantization parameters Qp'Cb, Qp'Cr, and Qp'CbCr may be derived.

[0204] The variable luma position (xQg, yQg) may indicate the position of the top left luma sample of the current quantization group corresponding to the top left sample of the current picture. Herein, the horizontal position xQg and the vertical position yQg may be set equal to the values ​​of the variables CuQgTopLeftX and CuQgTopLeftY, respectively. Fig. 20 In the coding tree syntax shown, variables CuQgTopLeftX and CuQgTopLeftY may be defined as predetermined values.

[0205] Here, the current quantization group can be a quadrilateral region within the coding tree block and can share the same qP Y_PRED Its width and height may be equal to the width and height of the coding tree node, with the top left luma sample position assigned to each of CuQgTopLeftX and CuQgTopLeftY.

[0206] When treeType is SINGLE_TREE or DUAL_TREE_LUMA, the luminance quantization parameter prediction value qP Y_PRED It can be derived as follows:

[0207] 1. Variable qP Y_PRED It can be derived as follows.

[0208] (Condition 1) When any of the following conditions is true, q PY_PRED The value can be set with SliceQp Y The same value (in this article, SliceQp Y Indicates the quantization parameter Qp of all slices in the picture Y Alternatively, qP Y_PRED The value of may be set to the luma quantization parameter Qp of the last luma coding basis of the quantization group immediately preceding in decoding order Y The value of .

[0209] -(Condition 1-1) When the current quantization group is the first quantization group in the slice

[0210] -(Condition 1-2) When the current quantization group is the first quantization group in the tile

[0211] - (Condition 1-3) When the current quantization group is the first quantization group in the CTB row in the tile and predetermined synchronization occurs (eg, when entropy_coding_sync_enabled_flag has a value of 1)

[0212] 2. Variable qP Y_A The value of can be derived as follows.

[0213] (Condition 2) When at least one of the following conditions is true, qP Y_A The value of qP can be set Y_PRED Alternatively, qP Y_A The value of may be set to the luma quantization parameter Qp of the coding basis for the luma coding block covering the luma sample position (xQg-1, yQg) Y The value of .

[0214] - (Condition 2-1) With respect to the block identified by the sample position (xCb, yCb), the block identified by the sample position (xQg-1, yQg) is not an available neighboring block,

[0215] - (Condition 2-2) When the CTB including the luma coding block covering the luma sample position (xQg-1, yQg) is different from the CTB including the current luma coding block at the luma sample position (xCb, yCb), for example, when all of the following conditions are true

[0216] -(Condition 2-2-1)(xQg-1)>>The value of CtbLog2SizeY is different from the value of (xCb)>>CtbLog2SizeY

[0217] -(Condition 2-2-2)(yQg)>>The value of CtbLog2SizeY is different from the value of (yCb)>>CtbLog2SizeY

[0218] 3. Variable qP Y_B The value of can be derived as follows.

[0219] (Condition 3) When at least one of the following conditions is true, qP Y_B The value of qP can be set Y_PRED Alternatively, qP Y_B The value of may be set to the luma quantization parameter Qp of the coding basis for the luma coding block covering the luma sample position (xQg, yQg-1). Y The value of .

[0220] - (Condition 3-1) With respect to the block identified by the sample position (xCb, yCb), when the block identified by the sample position (xQg, yQg-1) is not an available neighboring block,

[0221] - (Condition 3-2) When the CTB including the luma coding block covering the luma sample position (xQg, yQg-1) is different from the CTB including the current luma coding block at the luma sample position (xCb, yCb), for example, when all of the following conditions are true

[0222] -(Condition 3-2-1) (xQg) >> The value of CtbLog2SizeY is different from the value of (xCb) >> CtbLog2SizeY

[0223] -(Condition 3-2-2)(yQg-1)>>The value of CtbLog2SizeY is different from the value of (yCb)>>CtbLog2SizeY

[0224] 4. Luminance quantization parameter prediction value qP Y_PRED It can be derived as follows.

[0225] When all of the following conditions are true, qP Y_PRED The luma quantization parameter Qp may be set to include the coding basis of the luma coding block covering the luma sample position (xQg, yQg-1). Y .

[0226] - (Condition 3-1) With respect to the block identified by the sample position (xCb, yCb), the block identified by the sample position (xQg, yQg-1) is an available neighboring block

[0227] - When the current quantization group is the first quantization group in the CTB row in the tile

[0228] In addition, when all conditions are false, qP Y_PRED It can be derived as shown below.

[0229] [Formula 3]

[0230] Q Y_PRED =(qP Y_A +qP Y_B +1)>>1

[0231] Variable Qp Y It can be derived as shown below.

[0232] [Formula 4]

[0233] Q Y =((qP Y_PRED +CuQpDeltaVal+64+2*QpBdOffset)%(64+QpBdOffset))-QpBdOffset

[0234] Herein, CuQpDeltaVal indicates the difference between the luminance quantization parameter of the coding basis and its predicted value. Its value can be obtained from the bitstream. QpBdOffset indicates the luminance and chrominance quantization parameter range offset. QpBdOffset can be preset to a predetermined constant or obtained from the bitstream. For example, QpBdOffset can be calculated by multiplying a predetermined constant by the value of a syntax element indicating the bit depth of a luminance or chrominance sample. Luminance quantization parameter Qp' Y It can be derived as shown below.

[0235] [Formula 5]

[0236] Qp' Y =Qp Y +QpBdOffset

[0237] When the value of the variable ChromaArrayType indicating the type of the chroma array is not the first value (eg, 0) and treeType is SINGLE_TREE or DUAL_TREE_CHROMA, the following processing may be performed.

[0238] -When the value of treeType is DUAL_TREE_CHROMA, the variable Qp YThe value of may be set to the luma quantization parameter Qp corresponding to the luma coding basis covering the luma sample position (xCb+cbWidth / 2, yCb+cbHeight / 2) Y The value of is the same as the value of .

[0239] -Variable qP Cb ,qP Cr and qP CbCr It can be derived as shown below.

[0240] [Formula 6]

[0241] Q Chroma =Clip3(-QpBdOffset,63,Qp Y )

[0242] Q Cb =ChromaQpTable[0][qP Chroma ]

[0243] Q Cr =ChromaQpTable[1][qP Chroma ]

[0244] Q CbCr =ChromaQpTable[2][qP Chroma ]

[0245] Chroma quantization parameter Qp' of Cb and Cr components Cb and Qp' Cr And the chromaticity quantization parameter Qp' of the joint Cb-Cr coding CbCr It can be derived as shown below.

[0246] [Formula 7]

[0247] Qp' Cb =Clip3(-QpBdOffset,63,

[0248] Q Cb +pps_cb_qp_offset+slice_cb_qp_offset+CuQpOffset Cb )+QpBdOffset

[0249] Qp' Cr =Clip3(-QpBdOffset,63,

[0250] Q Cr +pps_cr_qp_offset+slice_cr_qp_offset+CuQpOffset Cr)+QpBdOffset

[0251] Qp' CbCr =Clip3(-QpBdOffset,63,

[0252] Q CbCr +pps_joint_cbcr_qp_offset+slice_joint_cbcr_qp_offset+CuQpOffset CbCr )+QpBdOffset

[0253] In the above formula, pps_cb_qp_offset and pps_cr_qp_offset are used to derive Qp' Cb and Qp' Cr The offset can be obtained from the bitstream of the picture parameter set. slice_cb_qp_offset and slice_cr_qp_offset are used to derive Qp' Cb and Qp' Cr CuQpOffset is the offset of the slice header and can be obtained from the slice header bitstream. Cb and CuQpOffset Cr is used to derive Qp' Cb and Qp' Cr The offset can be obtained from the bit stream of the transform basis.

[0254] In addition, for example, the dequantization process of the transform coefficient can be performed using the following parameters.

[0255] - Luma coordinates (xTbY, yTbY), which refer to the relative coordinates of the upper left sample of the current luma transform block relative to the upper left luma sample of the current picture

[0256] - variable nTbW, which indicates the width of the transform block

[0257] - variable nTbH which indicates the height of the transform block

[0258] - The variable cIdx, which indicates the color component of the current block

[0259] The output of this process may be an array d of scaled transform coefficients. Herein, the size of array d may be (nTbW) x (nTbH). The individual elements making up this may be identified as d[x][y].

[0260] To this end, the quantization parameter qP may be derived as follows: When cIdx has a value of 0, qP may be derived as shown in the following equation.

[0261] [Formula 8]

[0262] qP=Qp' Y

[0263] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, it can be derived as shown in the following equation.

[0264] [Formula 9]

[0265] qP=Qp' CbCr

[0266] Alternatively, when cIdx has a value of 1, qP may be derived as shown below.

[0267] [Formula 10]

[0268] qP=Qp' Cb

[0269] Alternatively, when cIdx has a value of 2, qP can be derived as shown in the following equation.

[0270] [Formula 11]

[0271] qP=Qp' Cr

[0272] Thereafter, the quantization parameter qP may be updated as follows. In addition, the variables rectNonTsFlag and bdShift may be derived as follows. For example, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0 (e.g., when the transform of the current transform block is not skipped), it may be derived as shown in the following formula. Herein, a first value (e.g., 0) of transform_skip_flag may indicate whether the transform is omitted as determined by another syntax element. A second value (e.g., 1) of transform_skip_flag may indicate that the transform is omitted (e.g., skipped).

[0273] [Formula 12]

[0274] qP=qP-(cu_act_enabled_flag[xTbY][yTbY]?5:0)

[0275] rectNonTsFlag=0

[0276] bdShift=10

[0277] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1 (eg, skipping the transformation of the current transform block), it may be derived as shown in the following equation.

[0278] [Formula 13]

[0279] qP=Max(QpPrimeTsMin,qP)-(cu_act_enabled_flag[xTbY][yTbY]?5:0)

[0280] rectNonTsFlag=(((Log2(nTbW)+Log2(nTbH))&1)==1

[0281] bdShift=BitDepth+(rectNonTsFlag?1:0)+((Log2(nTbW)+Log2(nTbH)) / 2)-+pic_dep_quant_enabled_flag

[0282] Herein, QpPrimeTsMin may indicate the minimum quantization parameter value allowed when the transform skip mode is applied. This may be determined as a predetermined constant or may be derived from a syntax element of the bitstream related thereto.

[0283] Herein, the suffixes Y, Cb, and Cr may denote G, B, and R color components in an RGB color model or Y, Cg, and Co color components in a YCgCo color model.

[0284] Quantization / Dequantization

[0285] As described above, the quantizer of the encoding device may derive quantized transform coefficients by applying quantization to the transform coefficients. The dequantizer of the encoding device or the dequantizer of the decoding device may derive transform coefficients by applying dequantization to the quantized transform coefficients.

[0286] In the encoding and decoding of video / still images, the quantization ratio can be changed, and the compression ratio can be adjusted using the changed quantization ratio. From the perspective of implementation, in consideration of complexity, a quantization parameter (QP) can be used instead of directly using the quantization ratio. For example, a quantization parameter having an integer value of 0 to 63 can be used, and each quantization parameter value can correspond to an actual quantization ratio. In addition, the quantization parameter QP of the luminance component (luminance sample) Y and the quantization parameter QP of the chroma component (chroma samples) C Can be set to be different from each other.

[0287] In the quantization process, the transform coefficient C may be input and divided by the quantization ratio Qstep, and based on this, the quantized transform coefficient C' may be obtained. In this case, considering the computational complexity, the quantization ratio may be multiplied by the scale to become an integer form, and a shift operation may be performed according to a value corresponding to the scale value. Based on the multiplication of the quantization ratio and the scale value, the quantization scale may be derived. That is, the quantization scale may be derived according to the QP. The quantization scale may be applied to the transform coefficient C, and based on this, the quantized transform coefficient C' may be derived.

[0288] The dequantization process is the inverse process of the quantization process. The quantized transform coefficient C' can be multiplied by the quantization ratio Qstep, and based on this, the reconstructed transform coefficient C" can be obtained. In this case, the level scale can be derived according to the quantization parameter. The level scale can be applied to the quantized transform coefficient C', and based on this, the reconstructed transform coefficient C" can be derived. Due to losses in the transformation and / or quantization process, the reconstructed transform coefficient C" may be slightly different from the original transform coefficient C. Therefore, the encoding device can perform dequantization in the same manner as the decoding device.

[0289] In addition, an adaptive frequency weighted quantization technique that adjusts the quantization strength according to the frequency can be applied. The adaptive frequency weighted quantization technique is a method of applying a quantization strength that varies with frequency. In the adaptive frequency weighted quantization, the quantization strength that varies with frequency can be applied by using a predefined quantization scaling matrix. That is, the above-mentioned quantization / dequantization process can be further performed based on the quantization scaling matrix. For example, different quantization scaling matrices can be used according to the size of the current block and / or whether the prediction mode applied to the current block to generate the residual signal of the current block is inter-frame prediction or intra-frame prediction. The quantization scaling matrix can be referred to as a quantization matrix or a scaling matrix. The quantization scaling matrix can be predefined. In addition, for frequency adaptive scaling, the frequency-specific quantization scaling information of the quantization scaling matrix can be constructed / encoded by the encoding device and can be notified to the decoding device with a signal. The frequency-specific quantization scaling information can be referred to as quantization scaling information. The frequency-specific quantization scaling information may include scaling list data (scaling_list_data). The (modified) quantization scaling matrix can be derived based on the scaling list data. In addition, the frequency-specific quantization scaling information may include a flag information indicating whether there is scaling list data. Alternatively, when the scaling list data is signaled at a high level (eg, SPS), information indicating whether the scaling list data is modified at a lower level (eg, PPS or tile group header, etc.) is also included.

[0290] Transformation / Inverse Transformation

[0291] As described above, the encoding device can derive a residual block (residual sample) based on a block (prediction sample) predicted by intra / inter / IBC prediction, and can derive a quantized transform coefficient by applying transform and quantization to the derived residual sample. By including in the residual coding syntax, information about the quantized transform coefficient (residual information) can be encoded and output in the form of a bitstream. The decoding device can obtain information about the quantized transform coefficient (residual information) from the bitstream, and can derive the quantized transform coefficient by performing decoding. The decoding device can derive the residual sample by dequantization / inverse transform based on the quantized transform coefficient. As described above, quantization / dequantization or transform / inverse transform or both can be omitted. When the transform / inverse transform is omitted, the transform coefficient can be referred to as a coefficient or a residual coefficient, or it can still be referred to as a transform coefficient for the sake of expression consistency. Whether the transform / inverse transform is omitted can be signaled based on a transform skip flag (e.g., transform_skip_flag).

[0292] Transformation / inverse transformation may be performed based on a transformation kernel. For example, a multiple transform selection (MTS) scheme for performing transformation / inverse transformation may be applied. In this case, some of a plurality of transform kernel sets may be selected and applied to the current block. The transform kernel may be referred to as various terms such as a transform matrix, a transform type, etc. For example, a transform kernel set may refer to a combination of a vertical direction transform kernel (vertical transform kernel) and a horizontal direction transform kernel (horizontal transform kernel).

[0293] Transformation / inverse transformation can be performed on a per CU or TU basis. That is, transformation / inverse transformation can be applied to residual samples in a CU or residual samples in a TU. The CU size and the TU size may be the same, or multiple TUs may exist in a CU area. In addition, the CU size may generally refer to the luminance component (sample) CB size. The TU size may generally refer to the luminance component (sample) TB size. The chrominance component (sample) CB or TB size may be derived based on the luminance component (sample) CB or TB size based on the component ratio according to the color format (chrominance format, for example, 4:4:4, 4:2:2, 4:2:0, etc.). The TU size may be derived based on maxTbSize. For example, when the CU size is greater than maxTbSize, multiple TUs (TBs) with maxTbSize may be derived from the CU, and transformation / inverse transformation may be performed on a per TU (TB) basis. maxTbSize may be considered when determining whether to apply various intra-frame prediction types such as ISP. Information about maxTbSize may be predetermined. Alternatively, information about maxTbSize may be generated and encoded by the encoding device and signaled to the encoding device.

[0294] Entropy Coding

[0295] As mentioned above Figure 2 As described, some or all of the video / image information may be entropy encoded by entropy encoder 190. Figure 3 Some or all of the described video / image information may be entropy decoded by the entropy decoder 310. In this case, the video / image information may be encoded / decoded on a per-syntax element basis. In this document, the encoding / decoding of the information may include encoding / decoding performed by the method described in this paragraph.

[0296] Fig.21 A block diagram of CABAC for encoding a syntax element is shown. In the encoding process of CABAC, first, when the input signal is a syntax element of a non-binary value, the input signal is converted into a binary value by binarization. When the input signal is already a binary value, the input signal bypasses the binarization. Herein, each binary number 0 or 1 that constitutes a binary value may be referred to as a bin. For example, when the binary string (bin string) after binarization is 110, each of 1, 1, and 0 is referred to as a bin. The bin of a syntax element may refer to the value of the syntax element.

[0297] The binarized bin can be input to a conventional coding engine or a bypass coding engine. The conventional coding engine can assign the context model of the application probability value to the corresponding bin, and can encode the bin based on the assigned context model. After encoding each bin, the conventional coding engine can update the probability model of the bin. The bin encoded in this way can be called a context coding bin. The bypass coding engine can omit the process for estimating the probability of the input bin and the process for updating the probability model applied to the bin after encoding. In the case of a bypass coding engine, instead of assigning a context, the input bin is encoded by applying a uniform probability distribution (e.g., 50:50), thereby improving the coding rate. The bin encoded in this way can be called a bypass bin. The context model can be assigned and updated for each bin to be context encoded (conventional encoding), and the context model can be indicated based on ctxidx or ctxInc. ctxidx can be derived based on ctxInc. Specifically, for example, the context index (ctxidx) indicating the context model of each conventionally coded bin can be derived as the sum of a context index increment (ctxInc) and a context index offset (ctxIdxOffset). In this article, ctxInc that varies with bin can be derived. ctxIdxOffset can be represented by the lowest value of ctxIdx. The lowest value of ctxIdx can be called the initial value (initValue) of ctxIdx. ctxIdxOffset is a value generally used to distinguish from context models of other syntax elements, and the context model of a syntax element can be distinguished / derived based on ctxinc.

[0298] During entropy coding, it is determined whether to perform coding through a regular coding engine or a bypass coding engine, and the coding path can be switched. Entropy decoding can perform the same process as entropy coding in reverse order.

[0299] For example, you can Fig. 22 and Fig.23 The above entropy coding is performed as in Fig. 22 and Fig.23 , the encoding device (entropy encoder) can perform an entropy encoding process on the image / video information. The image / video information may include segmentation related information, prediction related information (e.g., inter / intra prediction classification information, intra prediction mode information, and inter prediction mode information), residual information, and in-loop filtering related information, or may include various syntax elements related thereto. Entropy encoding may be performed on a per-syntax element basis. Fig. 22 Steps S2210 to S2220 can be performed by the above Figure 2 The encoding device is performed by the entropy encoder 190.

[0300] In step S2210, the encoding device may perform binarization on the target syntax element. Herein, the binarization may be based on various binarization methods such as truncated Rice binarization and fixed length binarization, and the binarization method for the target syntax element may be predefined. The binarization process may be performed by the binarization unit 191 in the entropy encoder 190.

[0301] In step S2220, the encoding device may perform entropy encoding on the target syntax element. The encoding device may perform encoding based on conventional encoding (based on context) or based on bypass encoding on the bin string of the target syntax element based on an entropy encoding scheme such as context adaptive arithmetic coding (CABAC) or context adaptive variable length coding (CAVLC). The output may be included in a bitstream. The entropy encoding process may be performed by an entropy encoding processor 192 in an entropy encoder 190. As described above, the bitstream may be transmitted to a decoding device via a (digital) storage medium or a network.

[0302] Reference Fig.24 and Fig.25 , the decoding device (entropy decoder) can decode the encoded image / video information. The image / video information may include segmentation related information, prediction related information (e.g., inter / intra prediction classification information, intra prediction mode information, and inter prediction mode information), residual information, and in-loop filtering related information, or may include various syntax elements related thereto. Entropy coding may be performed on a per-syntax element basis. Steps S2410 to S2420 may be performed by the above Figure 3 The entropy decoder 210 of the decoding device is executed.

[0303] In step S2410, the decoding device may perform binarization on the target syntax element. Herein, binarization may be based on various binarization methods such as truncated Rice binarization and fixed length binarization, and the binarization method for the target syntax element may be predefined. The decoding device may derive an available bin string (bin string candidate) of an available value of the target syntax element through the binarization process. The binarization process may be performed by the binarization unit 211 in the entropy decoder 210.

[0304] In step S2420, the decoding device can perform entropy decoding on the target syntax element. While sequentially decoding and parsing the various bins of the target syntax element from the input bits in the bitstream, the decoding device can compare the derived bin string and the available bin string of the syntax element. When the derived bin string is the same as one of the available bin strings, the value corresponding to the bin string can be derived as the value of the syntax element. If not, the next bit in the bitstream is further parsed and the above process is performed again. Through this process, the start bit or end bit of the specific information (specific syntax element) in the bitstream is not used, but the variable length bit is used to signal the information. Thus, relatively few bits are assigned for low values, thereby improving the overall coding efficiency.

[0305] The decoding device may perform context-based or bypass-based decoding on each bin in the bin string from the bitstream based on an entropy coding scheme such as CABAC or CAVLC. The entropy decoding process may be performed by an entropy decoding processor 212 in the entropy decoder 210. The bitstream may include various types of information for image / video decoding as described above. As described above, the bitstream may be transmitted to the decoding device via a (digital) storage medium or a network.

[0306] In this document, a table including syntax elements (syntax table) may be used for signaling information from an encoding device to a decoding device. The order of syntax elements in the table including syntax elements used in this document may refer to the parsing order of syntax elements from a bitstream. The encoding device may construct and encode the syntax table so that the decoding device parses the syntax elements in the parsing order. The decoding device may parse and decode the syntax elements of the syntax table from the bitstream in the parsing order, and thus may obtain the values ​​of the syntax elements.

[0307] General image / video encoding process

[0308] In image / video coding, the pictures constituting the image / video can be encoded / decoded according to a series of decoding orders. The picture order corresponding to the output order of the decoded pictures can be set to be different from the decoding order, and based on this, forward prediction and backward prediction can be performed when performing inter-frame prediction.

[0309] Fig.26 An example of an exemplary picture decoding process to which the embodiments of this document are applicable is shown. Fig.26 In step S2610, the above reference Figure 3The entropy decoder 210 of the decoding device described in the present invention may be performed by the entropy decoder 210 of the decoding device described in the present invention. Step S2620 may be performed by a prediction unit including an intra prediction unit 265 and an inter prediction unit 260. Step S2630 may be performed by a residual processor including a dequantizer 220 and an inverse transformer 230. Step S2640 may be performed by an adder 235. Step S2650 may be performed by a filter 240. Step S2610 may include the information decoding process described in this document. Step S2620 may include the inter / intra prediction process described in this document. Step S2630 may include the residual processing process described in this document. Step S2640 may include the block / picture reconstruction process described in this document. Step S2650 may include the in-loop filtering process described in this document.

[0310] Reference Fig.26 , as mentioned above Figure 3 As described, the picture decoding process may schematically include a process of obtaining image / video information from a bit stream (by decoding) in step S2610, a process of reconstructing a picture in steps 2620 to S2640, and a process of performing in-loop filtering on the reconstructed picture in step S2650. The picture reconstruction process may be performed based on the prediction samples and residual samples obtained by the inter / intra prediction in step S2620 and the residual processing (dequantization and inverse transformation of quantized transform coefficients) in step S2630 described in this document. The modified reconstructed picture may be generated by an in-loop filtering process for the reconstructed picture generated by the picture reconstruction process. The modified reconstructed picture may be output as a decoded picture and may be stored in a decoded picture buffer or memory 250 of a decoding device to be used as a reference picture in an inter-frame prediction process later when decoding the picture. In some cases, the in-loop filtering process may be omitted. In this case, the reconstructed picture may be output as a decoded picture and may be stored in a decoded picture buffer or memory 250 of a decoding device to be used as a reference picture in an inter-frame prediction process later when decoding the picture. As described above, the in-loop filtering process in step S2650 may include a deblocking filtering process, a sample adaptive offset (SAO) process, an adaptive loop filter (ALF) process, and / or a bilateral filter process. Some or all of them may be omitted. In addition, one or some of the deblocking filtering process, the sample adaptive offset (SAO) process, the adaptive loop filter (ALF) process, and the bilateral filter process may be applied sequentially, or all of them may be applied sequentially. For example, a deblocking filtering process may be applied to the reconstructed picture, and then the SAO process may be performed. Alternatively, for example, a deblocking filtering process may be applied to the reconstructed picture, and then the ALF process may be performed. This may be performed in the same manner as in the encoding device.

[0311] Fig. 27An example of an exemplary picture encoding process to which the embodiments of this document are applicable is shown. Fig. 27 In step S2710, the above reference Figure 2 The intra prediction unit 185 or the prediction unit 180 of the encoding device described in the present invention may be performed by the prediction unit. Step S2720 may be performed by a residual processor including the transformer 120 and / or the quantizer 130. Step S2730 may be performed by the entropy encoder 190. Step S2710 may include the inter / intra prediction process described in this document. Step S2720 may include the residual processing process described in this document. Step S2730 may include the information encoding process described in this document.

[0312] Reference Fig. 27 , as mentioned above Figure 2 As described, the picture encoding process may schematically include a process of encoding information (e.g., prediction information, residual information, and segmentation information) for picture reconstruction and outputting the information in the form of a bitstream, a process of generating a reconstructed picture of the current picture, and a process of applying in-loop filtering to the reconstructed picture (optional). The encoding device may derive (modified) residual samples from the quantized transform coefficients through a dequantizer 140 and an inverse transformer 150, and may generate a reconstructed picture based on the prediction samples and (modified) residual samples output in step S2710. The generated reconstructed picture may be the same as the reconstructed picture generated by the above-mentioned decoding device. An in-loop filtering process may be performed on the reconstructed picture to generate a modified reconstructed picture. The modified reconstructed picture may be stored in a decoded picture buffer or memory 170. Similar to the case in the decoding device, the modified reconstructed picture may be used as a reference picture in an inter-frame prediction process when encoding the picture later. As described above, in some cases, some or all of the in-loop filtering process may be omitted. When the in-loop filtering process is performed, (in-loop) filtering related information (parameters) may be encoded by an entropy encoder 190 and output in the form of a bitstream. The decoding device may perform an in-loop filtering process in the same manner as the encoding device based on the filtering-related information.

[0313] Through this in-loop filtering process, noise (such as block artifacts and ringing artifacts) generated during image / video encoding can be reduced, and subjective / objective visual quality can be improved. In addition, both the encoding device and the decoding device perform the in-loop filtering process so that the encoding device and the decoding device can derive the same prediction result, the reliability of picture encoding can be improved, and the amount of data to be sent for picture encoding can be reduced.

[0314] As described above, the picture reconstruction process can be performed in a decoding device as well as an encoding device. Reconstructed blocks can be generated based on individual blocks based on intra-frame prediction / inter-frame prediction, and a reconstructed picture including the reconstructed blocks can be generated. When the current picture / slice / patchwork group is an I picture / slice / patchwork group, the blocks included in the current picture / slice / patchwork group can be reconstructed based only on intra-frame prediction. In addition, when the current picture / slice / patchwork group is a P or B picture / slice / patchwork group, the blocks included in the current picture / slice / patchwork group can be reconstructed based on intra-frame prediction or inter-frame prediction. In this case, inter-frame prediction can be applied to some blocks in the current picture / slice / patchwork group, and intra-frame prediction can be applied to some remaining blocks. The color components of the picture may include a luminance component and a chrominance component. Unless explicitly limited in this document, the methods and embodiments proposed in this document may be applied to luminance components and chrominance components.

[0315] Examples of coding hierarchies and structures

[0316] The encoded video / image according to this document may be processed according to, for example, the coding hierarchy and structure to be described later.

[0317] Fig.28 It is a view showing the hierarchical structure of a coded image. The coded image can be divided into a video coding layer (VCL) that handles the image and its own decoding process, a subsystem for sending and storing coded information, and a network abstraction layer (NAL) that exists between the VCL and the subsystem and is responsible for the network adaptation function.

[0318] In the VCL, VCL data including compressed image data (slice data) can be generated. Alternatively, a parameter set including information such as a picture parameter set (PPS), a sequence parameter set (SPS), and a video parameter set (VPS) or a supplemental enhancement information (SEI) message additionally required for image decoding processing can be generated.

[0319] In NAL, header information (NAL unit header) is added to the raw byte sequence payload (RBSP) generated in the VCL so that the NAL unit can be generated. Herein, RBSP refers to slice data, parameter sets, and SEI messages generated in the VCL. The NAL unit header may include NAL unit type information specified according to the RBSP data included in the NAL unit.

[0320] As shown in the figure, the NAL unit can be divided into a VCL NAL unit and a non-VCL NAL unit according to the RBSP generated in the VCL. A VCL NAL unit may refer to a NAL unit including information about an image (slice data). A non-VCL NAL unit may refer to a NAL unit including information required for decoding an image (parameter set or SEI message).

[0321] In the case of attaching header information according to the data standard of the subsystem, the VCL NAL unit and the non-VCL NAL unit can be sent through the network. For example, the NAL unit can be converted into a data form according to a predetermined standard such as H.266 / VVC file format, real-time transport protocol (RTP), transport stream (TS), and the resulting data can be sent through various networks.

[0322] As described above, regarding the NAL unit, the NAL unit type may be specified according to the RBSP data structure included in the NAL unit, and information on the NAL unit type may be stored in the NAL unit header and signaled.

[0323] For example, depending on whether the NAL unit includes information about an image (slice data), it is roughly classified into a VCL NAL unit type and a non-VCL NAL unit type. The VCL NAL unit type can be classified according to the characteristics and type of the picture included in the VCL NAL unit, and the non-VCL NAL unit type can be classified according to the type of parameter set.

[0324] The following lists NAL unit types specified according to the type of parameter sets included in the non-VCL NAL unit type as examples.

[0325] -APS (Adaptation Parameter Set) NAL unit: A NAL unit that includes APS

[0326] -DPS (Decoding Parameter Set) NAL unit: A NAL unit that includes a DPS

[0327] -VPS (Video Parameter Set) NAL unit: A NAL unit that includes a VPS

[0328] -SPS (Sequence Parameter Set) NAL unit: A NAL unit that includes an SPS

[0329] -PPS (Picture Parameter Set) NAL unit: A NAL unit that includes a PPS

[0330] The above-mentioned NAL unit type may have syntax information of the NAL unit type, and the syntax information may be stored in the NAL unit header and signaled. For example, the syntax information may be nal_unit_type, and the NAL unit type may be specified by the nal_unit_type value.

[0331] The slice header (slice header syntax) may include information / parameters that are commonly applicable to the slice. APS (APS syntax) or PPS (PPS syntax) may include information / parameters that are commonly applicable to one or more slices or pictures. SPS (SPS syntax) may include information / parameters that are commonly applicable to one or more sequences. VPS (VPS syntax) may include information / parameters that are commonly applicable to multiple layers. DPS (DPS syntax) may include information / parameters that are commonly applicable to the entire video. DPS may include information / parameters related to the concatenation of coded video sequences (CVS). In this document, a high-level syntax (HLS) may include at least one selected from the group consisting of APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, and slice header syntax.

[0332] In this document, image / video information encoded by an encoding device and notified to a decoding device in the form of a bitstream by a signal may include information related to segmentation within a picture, intra-frame / inter-frame prediction information, residual information and in-loop filtering information, as well as information included in a slice header, information included in an APS, information included in a PPS, information included in an SPS and / or information included in a VPS.

[0333] Overview of Block Differential Pulse Code Modulation (BDPCM)

[0334] The image encoding device and the image decoding device according to the embodiment can perform differential encoding of the residual signal. For example, the image encoding device can encode the residual signal by subtracting the prediction signal from the residual signal of the current block, and the image decoding device can decode the residual signal by adding the prediction signal to the residual signal of the current block. The image encoding device and the image decoding device according to the embodiment can perform differential encoding of the residual signal by applying the following BDPCM.

[0335] The BDPCM according to the present disclosure may be performed in a quantized residual domain. The quantized residual domain may include a quantized residual signal (or a quantized residual coefficient), and when the BDPCM is applied, the transformation of the quantized residual signal may be skipped. For example, when the BDPCM is applied, the transformation of the residual signal may be skipped and quantization may be performed. Alternatively, the quantized residual domain may include a quantized transform coefficient.

[0336] In an embodiment of applying BDPCM, the image encoding device may derive a residual block of a current block predicted in an intra prediction mode and quantize the residual block, thereby deriving a residual block. When a differential encoding mode of a residual signal is performed on the current block, the image encoding device may perform differential encoding on the residual block to derive a modified residual block. In addition, the image encoding device may encode differential encoding mode information specifying a differential encoding mode of a residual signal and the modified residual block, thereby generating a bitstream.

[0337] More specifically, when BDPCM is applied to the current block, a predicted block (prediction block) including a prediction sample of the current block can be generated by intra prediction. In this case, the intra prediction mode for performing intra prediction can be notified by a bit stream signal and can be derived based on the prediction direction of the BDPCM described below. In addition, in this case, the intra prediction mode can be determined as one of a vertical prediction direction mode or a horizontal prediction direction mode. For example, when the prediction direction of BDPCM is a horizontal direction, the intra prediction mode can be determined as a horizontal prediction direction mode, and the prediction block of the current block can be generated by intra prediction in the horizontal direction. Alternatively, when the prediction direction of BDPCM is a vertical direction, the intra prediction mode can be determined as a vertical prediction direction mode, and the prediction block of the current block can be generated by intra prediction in the vertical direction. When applying intra prediction in the horizontal direction, the value of the pixel adjacent to the left side of the current block can be determined as the predicted sample value of the sample included in the corresponding row of the current block. When intra prediction in the vertical direction is applied, the value of the pixel adjacent to the top of the current block can be determined as the predicted sample value of the sample included in the corresponding column of the current block. When BDPCM is applied to a current block, a method of generating a prediction block of the current block may be equally performed in an image encoding apparatus and an image decoding apparatus.

[0338] When BDPCM is applied to the current block, the image encoding device may generate a residual block including residual samples of the current block by subtracting the prediction sample from the current block. The image encoding device may quantize the residual block and then encode the difference (or increment) between the quantized residual sample and the predictor of the quantized residual sample. The image decoding device may generate a quantized residual block of the current block by obtaining the quantized residual sample of the current block based on the predictor and the difference reconstructed from the bitstream. Thereafter, the image decoding device may dequantize the quantized residual block and then add it to the prediction block to reconstruct the current block.

[0339] Fig.29 is a view illustrating a method of encoding residual samples of BDPCM according to the present disclosure. Fig.29 The residual block may be generated by subtracting the prediction block from the current block in an image encoding device. Fig.29The quantized residual block can be generated by quantizing the residual block by the image encoding device. Fig.29 In, r i,j Specifies the value of the residual sample at the (i, j) coordinate in the current block. When the size of the current block is M×N, the value i can be 0 to M-1 (inclusive). In addition, the value j can be 0 to N-1 (inclusive). For example, the residual can refer to the difference between the original block and the predicted block. For example, r i,j It can be derived by subtracting the value of the predicted sample from the value of the original sample at coordinate (i, j) in the current block. i,j It can be the prediction residual after performing horizontal intra prediction or vertical intra prediction using samples that are not filtered from the upper or left boundary samples. In horizontal intra prediction, the values ​​of the left neighboring pixels are copied along the rows that intersect the prediction block. In vertical intra prediction, the upper neighboring rows are copied to separate rows of the prediction block.

[0340] exist Fig.29 In the i,j ) refers to the value of the quantized residual sample at coordinate (i, j) in the current block. For example, Q(r i,j ) can refer to r i,j Quantified value of .

[0341] right Fig.29 The prediction of BDPCM is performed on the quantized residual samples, and a modified quantized residual block R' having a size of M×N including the modified quantized residual samples r' may be generated.

[0342] When the prediction direction of BDPCM is horizontal, the value r' of the modified quantized residual sample at coordinate (i, j) in the current block i,j It can be calculated as shown below.

[0343] [Formula 14]

[0344]

[0345] As shown in Equation 14, when the prediction direction of BDPCM is horizontal, the value Q(r 0,j ) is assigned to the value r' at coordinate (0,j) as is 0,j . The values ​​of other coordinates (i,j) r' i,j can be derived as the value Q(r) of the quantized residual sample at coordinate (i, j) i,j ) and the value Q(r) of the quantized residual sample at coordinate (i-1, j) i-1,j That is, instead of the value Q(r) of the quantized residual sample at coordinate (i, j), i,j ) is encoded by using the value Q(r i-1,j) is used as the difference calculated as the predicted value and is derived as the modified quantized residual sample value r' i,j , then for the value r' i,j to encode.

[0346] When the prediction direction of BDPCM is vertical, the value of the modified quantized residual sample (r' i,j ) can be calculated as shown below.

[0347] [Formula 15]

[0348]

[0349] As shown in Equation 15, when the prediction direction of BDPCM is vertical, the value Q(r i,0 ) is assigned to the value r' at coordinate (i,0) as is i,0 . The values ​​of other coordinates (i,j) r' i,j can be derived as the value Q(r) of the quantized residual sample at coordinate (i, j) i,j ) and the value Q(r) of the quantized residual sample at coordinate (i, j-1) i,j-1 That is, instead of the value Q(r) of the quantized residual sample at coordinate (i, j), i,j ) is encoded by using the value Q(r i,j-1 ) is used as the difference calculated as the predicted value and is derived as the modified quantized residual sample value r' i,j , then for the value r' i,j to encode.

[0350] As described above, the process of modifying the current quantized residual sample value by using the nearby quantized residual sample values ​​as the prediction value may be referred to as BDPCM prediction.

[0351] Finally, the image encoding device may encode the modified quantized residual block including the modified quantized residual samples, and may transmit the resulting block to the image decoding device.Herein, as described above, the transformation of the modified quantized residual block is not performed.

[0352] Fig.30 is a view showing a modified quantized residual block generated by performing BDPCM of the present disclosure.

[0353] exist Fig.30 , the horizontal BDPCM shows a modified quantized residual block generated according to Equation 14 when the prediction direction of the BDPCM is the horizontal direction. In addition, the vertical BDPCM shows a modified quantized residual block generated according to Equation 15 when the prediction direction of the BDPCM is the vertical direction.

[0354] Fig.31 is a flowchart illustrating a process of encoding a current block by applying BDPCM in an image encoding apparatus.

[0355] First, when the current block as the encoding target block is input in step S3110, prediction may be performed on the current block to generate a prediction block in step S3120. The prediction block in step S3120 may be an intra-frame prediction block, and the intra-frame prediction mode may be determined as described above. Based on the prediction block generated in step S3120, a residual block of the current block may be generated in step S3130. For example, the image encoding device may generate a residual block (the value of the residual sample) by subtracting the prediction block (the value of the prediction sample) from the current block (the value of the original sample). For example, by performing step S3130, a residual block may be generated. Fig.29 For the residual block generated in step S3130, quantization may be performed in step S3140 to generate a quantized residual block, and BDPCM prediction may be performed on the quantized residual block in step S3150. The quantized residual block generated as a result of performing step S3140 may be Fig.29 As a result of the BDPCM prediction in step S3150, Fig.30 The modified quantized residual block can be generated according to the prediction direction. Figure 29 to Figure 30 The description is made, so the detailed description thereof will be omitted. Thereafter, the image encoding apparatus may encode the modified quantized residual block at step S3160 to generate a bitstream. Herein, the transformation of the modified quantized residual block may be skipped.

[0356] Reference in image coding device Figure 29 to Figure 31 The described BDPCM operations may be performed in reverse by an image decoding device.

[0357] Fig.32 is a flowchart illustrating a process of reconstructing a current block by applying BDPCM in an image decoding apparatus.

[0358] In step S3210, the image decoding device may obtain information (image information) required to reconstruct the current block from the bitstream. The information required to reconstruct the current block may include information about the prediction of the current block (prediction information) and information about the residual of the current block (residual information). The image decoding device may perform prediction on the current block based on the information about the current block, and may generate a prediction block in step S3220. The prediction of the current block may be intra-frame prediction, and its detailed description is the same as that described above with reference to Fig.31 Same as described. Fig.32, step S3220 of generating a prediction block of the current block is shown to be performed before steps S3230 to S3250 of generating a residual block of the current block. However, no limitation is imposed thereon. After generating the residual block of the current block, the prediction block of the current block may be generated. Alternatively, the residual block of the current block and the prediction block of the current block may be generated simultaneously.

[0359] The image decoding apparatus may generate a residual block of the current block by parsing the residual information of the current block from the bitstream at step S3230. The residual block generated at step S3230 may be Fig.30 The modified quantized residual block is shown.

[0360] The image decoding device can perform the following steps in step S3240: Fig.30 The modified quantized residual block performs BDPCM prediction to generate Fig.29 The BDPCM prediction in step S3240 is from Fig.30 The modified quantized residual block is generated Fig.29 The process of quantizing the residual block of the image encoding device corresponds to the inverse process of step S3150 performed by the image encoding device. For example, when the differential coding mode information (e.g., bdpcm_flag) obtained from the bitstream indicates a differential coding mode in which differential coding of the residual coefficient is performed when BDPCM is applied, the image decoding device performs differential coding on the residual block to derive the modified residual block. Using the residual coefficient to be modified and the predicted residual coefficient, the image decoding device can modify at least one residual coefficient to be modified among the residual coefficients in the residual block. The predicted residual coefficient can be determined based on the prediction direction indicated by the differential coding direction information (e.g., bdpcm_dir_flag) obtained from the bitstream. The differential coding direction information can indicate a vertical direction or a horizontal direction. The image decoding device can assign a value obtained by adding the residual coefficient to be modified and the predicted residual coefficient to the position of the residual coefficient to be modified. Herein, the predicted residual coefficient may be a coefficient that is immediately before and adjacent to the residual coefficient to be modified in terms of the order according to the prediction direction.

[0361] Hereinafter, the BDPCM prediction in step S3240 performed by the image decoding device will be described in more detail below. The decoding device can calculate the quantized residual sample Q (r i,j ). For example, when the prediction direction of BDPCM is the horizontal direction, the image decoding device may generate a quantized residual block from the modified quantized residual block by using Equation 16.

[0362] [Formula 16]

[0363]

[0364] As defined in Equation 16, the value Q(r i,j ) can be calculated by adding the values ​​of the modified quantized residual samples starting from coordinate (0,j) to coordinate (i,j).

[0365] Alternatively, using Equation 17 instead of Equation 16, the value Q(r) of the quantized residual sample at coordinate (i, j) can be calculated. i,j ).

[0366] [Formula 17]

[0367]

[0368] Equation 17 is the inverse process corresponding to Equation 14. According to Equation 17, the value Q(r) of the quantized residual sample at the coordinate (0, j) is 0,j ) is derived as the value r' of the modified quantized residual sample at coordinate (0, j) 0,j . Q(r i,j ) is derived as the value r' of the modified quantized residual sample at coordinate (i, j) i,j and the value Q(r i-1,j That is, by using the value Q(r) of the quantized residual sample at coordinate (i-1, j) i-1,j ) as the predicted value and take the difference r' i,j Add together to derive the quantized residual sample value Q(r i,j ).

[0369] When the prediction direction of the BDPCM is a vertical direction, the image decoding apparatus may generate a quantized residual block from the modified quantized residual block by using Equation 18.

[0370] [Formula 18]

[0371]

[0372] As defined in Equation 18, the value Q(r i,j ) can be calculated by adding the values ​​of the modified quantized residual samples starting from coordinate (i,0) to coordinate (i,j).

[0373] Alternatively, using Equation 19 instead of Equation 18, the value Q(r) of the quantized residual sample at coordinate (i, j) can be calculated. i,j ).

[0374] [Formula 19]

[0375]

[0376] Equation 19 is the inverse process corresponding to Equation 15. According to Equation 19, the value Q(r) of the quantized residual sample at the coordinate (i, 0) is i,0 ) is derived as the value r' of the modified quantized residual sample at coordinate (i,0) i,0 . Q(r i,j ) is derived as the value r' of the modified quantized residual sample at coordinate (i, j) i,j and the value Q(r i,j-1 That is, by using the value Q(r) of the quantized residual sample at coordinate (i, j-1) i,j-1 ) as the predicted value and take the difference r' i,j Add together to derive the quantized residual sample value Q(r i,j ).

[0377] When a quantized residual block composed of quantized residual samples is generated by executing step S3240 according to the above method, in step S3250, the image decoding device performs dequantization on the quantized residual block to generate a residual block of the current block. When BDPCM is applied, the transformation of the current block is skipped as described above. Therefore, the inverse transformation of the dequantized residual block can be skipped.

[0378] Thereafter, the image decoding apparatus may reconstruct the current block based on the prediction block generated in step S3220 and the residual block generated in step S3250 at step S3260. For example, the image decoding apparatus may reconstruct the current block (the value of the reconstructed sample) by adding the prediction block (the value of the prediction sample) and the residual block (the value of the residual sample). For example, the image decoding apparatus may reconstruct the current block (the value of the reconstructed sample) by adding the dequantized quantized sample Q -1 (Q(r i,j )) is added to the intra-frame block prediction value to generate a reconstructed sample value. Differential coding mode information indicating whether BDPCM is applied to the current block may be notified by a signal through the bitstream. In addition, when BDPCM is applied to the current block, differential coding direction information indicating the prediction direction of BDPCM may be notified by a signal through the bitstream. When BDPCM is not applied to the current block, the differential coding direction information may not be notified by a signal.

[0379] Figure 33 to Figure 35 is a view schematically showing a syntax for signaling information about BDPCM.

[0380] Fig.33is a view showing the syntax of a sequence parameter set according to an embodiment of signaling BDPCM information. In an embodiment, all SPS RBSPs included in at least one access unit (AU) having a value of 0 as a temporary ID (TemporalId) or provided by external means may be set to be used before being referenced in a decoding process. In addition, an SPS NAL unit including an SPS RBSP may be set to have the same nuh_layer_id as the nuh_layer_id of a PPS NAL unit that references the SPS NAL unit. In a CVS, all SPS NAL units having a specific sps_seq_parameter_set_id value may be set to have the same content. Fig.33 In the seq_parameter_set_rbsp() syntax of , the above-mentioned sps_transform_skip_enable_flag and the sps_bdpcm_enabled_flag described later are disclosed.

[0381] The syntax element sps_bdpcm_enabled_flag may indicate whether intra_bdpcm_flag is provided in the CU syntax for an intra coding unit. For example, a first value of sps_bdpcm_enabled_flag (e.g., 0) may indicate that intra_bdpcm_flag is not provided in the CU syntax for an intra coding unit. A second value of sps_bdpcm_enabled_flag (e.g., 1) may indicate that intra_bdpcm_flag is provided in the CU syntax for an intra coding unit. In addition, when sps_bdpcm_enabled_flag is not provided, the value of sps_bdpcm_enabled_flag may be set to a first value (e.g., 0).

[0382] Fig.34 2 is a diagram showing an embodiment of a syntax for signaling whether to apply restrictions on BDPCM. In an embodiment, a predetermined restriction condition in an encoding / decoding process may be signaled using a general_constraint_info() syntax. Fig.34The syntax of the sps_bdpcm_enabled_flag may signal a syntax element no_bdpcm_constraint_flag indicating whether the value of the above-mentioned sps_bdpcm_enabled_flag needs to be set to 0. For example, a first value (e.g., 0) of no_bdpcm_constraint_flag may indicate that such restriction is not applied. When the value of no_bdpcm_constraint_flag is a second value (e.g., 1), the value of sps_bdpcm_enabled_flag may be forced to be a first value (e.g., 0).

[0383] Fig.35 2 is a diagram showing an embodiment of a coding unit () syntax for signaling information about a BDPCM of a coding unit. Fig.35 As shown, the syntax elements intra_bdpcm_flag and intra_bdpcm_dir_flag may be signaled using the coding_unit() syntax. The syntax element intra_bdpcm_flag may indicate whether BDPCM is applied to the current luma coding block located at (x0, y0).

[0384] For example, the first value of intra_bdpcm_flag (e.g., 0) may indicate that BDPCM is not applied to the current luma coding block. The second value of intra_bdpcm_flag (e.g., 1) may indicate that BDPCM is applied to the current luma coding block. By indicating that BDPCM is applied, intra_bdpcm_flag may indicate whether to skip transform and whether to perform intra luma prediction mode according to intra_bdpcm_dir_flag (described later).

[0385] In addition, regarding x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1, the value of the above-mentioned variable BdpcmFlag[x][y] can be set to the value of intra_bdpcm_flag.

[0386] The syntax element intra_bdpcm_dir_flag may indicate the prediction direction of BDPCM. For example, a first value (eg, 0) of intra_bdpcm_dir_flag may indicate that the BDPCM prediction direction is a horizontal direction. A second value (eg, 1) of intra_bdpcm_dir_flag may indicate that the BDPCM prediction direction is a vertical direction.

[0387] Furthermore, with respect to x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1, the value of the variable BdpcmDir[x][y] may be set to the value of intra_bdpcm_dir_flag.

[0388] Intra prediction for chroma blocks

[0389] When intra prediction is performed on the current block, prediction of the luminance component block (luminance block) of the current block and prediction of the chrominance component block (chrominance block) can be performed. In this case, the intra prediction mode of the chrominance block can be set separately from the intra prediction mode of the luminance block.

[0390] For example, the intra prediction mode of the chroma block may be indicated based on the intra chroma prediction mode information. The intra chroma prediction mode information may be signaled in the form of a syntax element intra_chroma_pred_mode. For example, the intra chroma prediction mode information may indicate one of the following: a plane mode, a DC mode, a vertical mode, a horizontal mode, a derivation mode (DM), and a cross component linear model (CCLM) mode. Herein, the plane mode may refer to intra prediction mode 0, the DC mode may refer to intra prediction mode 1, the vertical mode may refer to intra prediction mode 26, and the horizontal mode may refer to intra prediction mode 10. DM may also be referred to as a direct mode. CCLM may also be referred to as a linear model (LM). The CCLM mode may include any one of the following: L_CCLM, T_CCLM, and LT_CCLM.

[0391] In addition, DM and CCLM are related intra prediction modes for predicting chrominance blocks by using information of luma blocks. DM may refer to a mode in which the same intra prediction mode as the intra prediction mode of the luma component is applied as the intra prediction mode of the chroma component. In addition, CCLM may refer to an intra prediction mode in which, in the process of generating a prediction block of a chroma block, a reconstructed sample of the luma block is subsampled, and then a sample derived by applying CCLM parameters α and β to the subsampled sample is used as a prediction sample of the chroma block.

[0392] Overview of the Cross-Component Linear Model (CCLM) model

[0393] As described above, the CCLM mode may be applied to the chrominance block. The CCLM mode is an intra-frame prediction mode using the correlation between the luminance block and the chrominance block corresponding to the luminance block, and is performed by deriving a linear model based on neighboring samples of the luminance block and neighboring samples of the chrominance block. In addition, the prediction sample of the chrominance block may be derived based on the derived linear model and the reconstructed sample of the luminance block.

[0394] Specifically, when the CCLM mode is applied to the current chrominance block, the parameters of the linear model can be derived based on the neighboring samples used for intra prediction of the current chrominance block and the neighboring samples used for intra prediction of the current luminance block. For example, the linear model of CCLM can be expressed based on the following formula.

[0395] [Formula 20]

[0396] pred c (i, j) = α·rec L ′(i,j)+β

[0397] Herein, predc(i,j) may refer to the predicted sample of the coordinate (i,j) of the current chroma block in the current CU. L '(i,j) may refer to the reconstructed sample of the coordinate (i,j) of the current luminance block in the CU. For example, rec L '(i,j) may refer to the downsampled reconstructed samples of the current luma block. The linear model coefficients α and β may be signaled or derived from neighboring samples.

[0398] Joint Coding of Residual (Joint CbCr)

[0399] In the encoding / decoding process according to the embodiment, the chroma residual can be encoded / decoded together. This can be referred to as joint coding of the residual or joint CbCr. Whether the joint coding mode of CbCr is applied (activated) can be signaled by the joint coding mode signaling flag tu_joint_cbcr_residual_flag signaled at the level of the transform basis. In addition, the selected coding mode can be derived by the chroma CBF. When the value of at least one chroma CBF of the transform basis is 1, the flag tu_joint_cbcr_residual_flag may be present. The chroma QP offset value indicates the difference between the universal chroma QP offset value signaled for the conventional chroma residual coding mode and the chroma QP offset value of the CbCr joint coding mode. The chroma QP offset value can be signaled by the PPS or slice header. The QP offset value can be used to derive the chroma QP value of the block using the joint chroma residual coding mode.

[0400] When mode 2 in the following table as the corresponding joint chroma coding mode is activated for the transform basis, its chroma QP offset can be added to the target luma-derived chroma QP (applied luma-derived chroma QP) while performing quantization and decoding of the transform basis.

[0401] For other modes (like modes 1 and 3 in the table below), the chroma QP can be derived in the same way as obtained for a common Cb or Cr block. This process of reconstructing the chroma residual (resCb and resCr) from the transform block can be selected according to the table below. When the current mode is activated, a single joint chroma residual block (resJointC[x][y] in the table below) is signaled, and the residual block resCb for Cb and the residual block resCr for Cr can be derived taking into account information such as tu_cbf_cb, tu_cbf_cr and CSign (the symbol value disclosed in the slice header).

[0402] In the encoding device, the joint chrominance component may be derived as follows. Depending on the joint encoding mode, resJointC{1,2} may be generated according to the following sequence. In the case of mode 2 (single residual with reconstruction Cb=C, Cr=CSign*C), the joint residual may be determined according to the following formula.

[0403] [Formula 21]

[0404] resJointC[x][y]=(resCb[x][y]+CSign*resCr[x][y]) / 2.

[0405] Alternatively, in the case of Mode 1 (single residual with reconstruction Cb=C, Cr=(CSign*C) / 2), the joint residual may be determined according to the following equation.

[0406] [Formula 22]

[0407] resJointC[x][y]=(4*resCb[x][y]+2*CSign*resCr[x][y]) / 5.

[0408] Alternatively, in case of mode 3 (single residual with reconstruction Cr=C, Cb=(CSign*C) / 2), the joint residual may be determined according to the following equation.

[0409] [Formula 23]

[0410] resJointC[x][y]=(4*resCr[x][y]+2*CSign*resCb[x][y]) / 5.

[0411] [Table 2]

[0412]

[0413] The above represents the reconstruction of the chroma residual. CSign refers to the symbol value +1 or -1 specified in the slice header. resJointC[][] refers to the transmitted residual. In this table, the mode refers to TuCResMode (described later). The three joint chroma coding modes in the table can be supported only for I slices. For P and B slices, only mode 2 can be supported. Therefore, for P and B slices, the syntax element tu_joint_cbcr_residual_flag can only be provided when the values ​​of two chroma cbfs (e.g., tu_cbf_cb and tu_cbf_cr) are 1. In addition, in the context modeling of tu_cbf_luma and tu_cbf_cb, the transform depth can be removed.

[0414] Implementation 1: QP update method using ACTQp_offset

[0415] As described above, an update of QP may be performed to apply ACT. The above update of QP has several problems. For example, when using the above method, different ACT Qp offsets cannot be set for each color component. In addition, the derived qP value may have a negative value. Therefore, in the following embodiment, a method of applying clipping to the Qp value derived based on the ACT QP offset value of the color component value is described.

[0416] In an embodiment, the quantization parameter qP may be derived as follows.

[0417] First, when cIdx has a value of 0, qP and ACT Qp offset can be derived as shown below.

[0418] [Equation 24]

[0419] qP=Qp' Y

[0420] ActQpOffset=5

[0421] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0422] [Formula 25]

[0423] qP=Qp' CbCr

[0424] ActQpOffset=5

[0425] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset may be derived as shown below.

[0426] [Equation 26]

[0427] qP=Qp'Cb

[0428] ActQpOffset=3

[0429] The quantization parameter qP can be updated as follows.

[0430] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be derived as shown in the following equation.

[0431] [Formula 27]

[0432] qP=Max(0,qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0433] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be derived as shown in the following equation.

[0434] [Equation 28]

[0435] qP=Max(0,Max(QpPrimeTsMin,qP)-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0436] In another embodiment, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be clipped using a value of QpPrimeTsMin instead of 0 as shown in the following equation.

[0437] [Equation 29]

[0438] qP=Max(QpPrimeTsMin,qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[0439] Furthermore, in another embodiment, the quantization parameter qP may be derived as follows.

[0440] First, when cIdx has a value of 0, qP and ACT Qp offset can be derived as shown below.

[0441] [Formula 30]

[0442] qP=Qp' Y

[0443] ActQpOffset=5

[0444] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0445] [Equation 31]

[0446] qP=Qp' CbCr

[0447] ActQpOffset=5

[0448] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset may be derived as shown below.

[0449] [Formula 32]

[0450] qP=Qp' Cb

[0451] ActQpOffset=5

[0452] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0453] [Formula 33]

[0454] qP=Qp' Cr

[0455] ActQpOffset=3

[0456] The quantization parameter qP can be updated as follows.

[0457] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be derived as shown in the following equation.

[0458] [Equation 34]

[0459] qP=Max(0,qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0460] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be derived as shown in the following equation.

[0461] [Formula 35]

[0462] qP=Max(0,Max(QpPrimeTsMin,qP)-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0463] In another embodiment, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be clipped using a value of QpPrimeTsMin instead of 0 as shown in the following equation.

[0464] [Equation 36]

[0465] qP=Max(QpPrimeTsMin,qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0466] Furthermore, in another embodiment, the quantization parameter qP may be derived as follows.

[0467] First, when cIdx has a value of 0, qP and ACT Qp offset can be derived as shown below.

[0468] [Equation 37]

[0469] qP=Qp' Y

[0470] ActQpOffset=-5

[0471] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0472] [Equation 38]

[0473] qP=Qp' CbCr

[0474] ActQpOffset=-5

[0475] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset may be derived as shown below.

[0476] [Equation 39]

[0477] qP=Qp' Cb

[0478] ActQpOffset=-5

[0479] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0480] [Formula 40]

[0481] qP=Qp' Cr

[0482] ActQpOffset=-3

[0483] The quantization parameter qP can be updated as follows.

[0484] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be derived as shown in the following equation.

[0485] [Formula 41]

[0486] qP=Max(0,qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0487] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be derived as shown in the following equation.

[0488] [Formula 42]

[0489] qP=Max(0,Max(QpPrimeTsMin,qP)+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0490] Furthermore, in another embodiment, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be clipped using a value of QpPrimeTsMin instead of 0 as shown in the following equation.

[0491] [Formula 43]

[0492] qP=Max(QpPrimeTsMin,qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0493] In the above description, Y, Cb, and Cr may represent three color components. For example, in ACT transformation, Y may correspond to C0. Cb may correspond to C1 or Cg. In addition, Cr may correspond to C2 or Co.

[0494] In addition, the ACTQpOffset values ​​-5, -5, and -3 of the three color components may be replaced with other values ​​or other variables.

[0495] Implementation 2: Signaling of QP offset adjustment in ACT

[0496] In the above embodiment, the ACT QP offset adjustment is fixed to -5, -5, and -3 for the Y, Cg, and Co components. In this embodiment, in order to provide greater flexibility to the ACT QP adjustment offset, a method of signaling the ACT QP offset will be described. The ACT QP offset can be signaled as a parameter in the PPS.

[0497] In an implementation, the Fig.36 The syntax table of qp_offset is used to signal qp_offset. Its syntax elements are as follows.

[0498] The syntax element pps_act_qp_offsets_present_flag may indicate whether there is a syntax element related to ACT QP offset in PPS. For example, pps_act_qp_offsets_present_flag may indicate whether the syntax elements pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset (to be described later) are signaled as PPS.

[0499] For example, a first value (eg, 0) of pps_act_qp_offsets_present_flag may indicate that pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset are not signaled through the PPS syntax table.

[0500] A second value (eg, 1) of pps_act_qp_offsets_present_flag may indicate that pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset are signaled through the PPS syntax table.

[0501] When pps_act_qp_offsets_present_flag is not provided from the bitstream, pps_act_qp_offsets_present_flag may be derived as a first value (e.g., 0). For example, when a flag indicating whether ACT is applied (e.g., sps_act_enabled_flag signaled in SPS) has a first value (e.g., 0) indicating that ACT is not applied, pps_act_qp_offsets_present_flag may be forced to have a first value (e.g., 0).

[0502] When the value of the syntax element cu_act_enabled_flag is the second value (e.g., 1) indicating that ACT is applied for the current coding basis, the syntax elements pps_act_y_qp_offset_plus5, pps_act_cb_qp_offset_plus5s, and pps_act_cr_qp_offset_plus3 may be used to determine the offsets of the quantization parameter values ​​qP applied to the luma, Cb, and Cr components, respectively. When the values ​​of pps_act_y_qp_offset_plus5, pps_act_cb_qp_offset_plus5, and pps_act_cr_qp_offset_plus3 do not exist in the bitstream, the respective values ​​may be set to 0.

[0503] According to the syntax element, the value of the variable PpsActQpOffsetY may be determined as pps_act_y_qp_offset_plus5 - 5. The value of the variable PpsActQpOffsetCb may be determined as pps_act_cb_qp_offset_plus5 - 5. In addition, the value of the variable PpsActQpOffsetCr may be determined as pps_act_cb_qp_offset_plus3 - 3.

[0504] Herein, ACT is not an orthogonal transform, so 5, 5, and 3 may be applied as constant offset values ​​to be subtracted. In an embodiment, for bitstream matching, the values ​​of PpsActQpOffsetY, PpsActQpOffsetCb, and PpsActQpOffsetCr may have values ​​in the range of -12 to 12. In addition, according to an embodiment, in addition to 5, 5, and 3, the Qp offset value may be replaced with other constant values ​​and used.

[0505] In another embodiment, a more flexible ACT_QP offset may be used to adjust the QP. In the following embodiment, an example of signaling the ACT QP offset in the bitstream is described. Therefore, the ACT QP offset may have a wider offset range. Therefore, the QP updated using the ACT QP offset is more likely to exceed the available range, so it is necessary to perform trimming on the upper and lower limits of the updated QP (a more detailed embodiment will be described later in Embodiments 6 and 7).

[0506] The variables PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, and PpsActQpOffsetCbCr indicating the ACT QP offset may be values ​​derived using the ACT QP offset signaled through the bitstream or a preset constant. For bitstream consistency, PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, and PpsActQpOffsetCbCr may have values ​​in the range of -12 to +12.

[0507] When a fixed value is not used, the value of the QP offset is signaled and has a value in the range of -12 to 12. In addition to clipping the lower limit of the derived QP value to avoid having a negative QP, it is also necessary to clip the upper limit of the derived QP value.

[0508] To prevent the value of qP from having a negative value, the lowest value of qP may be forced to 0. Alternatively, the lowest value of qP may be set to a value determined by a signaled syntax element. For example, to signal the lowest value of qP when the transform skip mode is applied, a syntax element QpPrimeTsMin indicating the value of qP applied when the transform skip mode is applied may be used. The maximum value of qP may be limited to an available maximum value of qP (e.g., 63) or a maximum available qP value determined according to a signaled syntax element.

[0509] In the embodiment according to the above, the quantization parameter qP may be derived as follows: First, when cIdx has a value of 0, qP and ACT Qp offset may be derived as shown in the following equation.

[0510] [Formula 44]

[0511] qP=Qp' Y

[0512] ActQpOffset=PpsActQpOffsetY

[0513] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0514] [Formula 45]

[0515] qP=Qp' CbCr

[0516] ActQpOffset=PpsActQpOffsetCbCr

[0517] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset may be derived as shown below.

[0518] [Formula 46]

[0519] qP=Qp' Cb

[0520] ActQpOffset=PpsActQpOffsetCb

[0521] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0522] [Formula 47]

[0523] qP=Qp' Cr

[0524] ActQpOffset=PpsActQpOffsetCr

[0525] In an embodiment, the quantization parameter qP may be updated as follows: When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be derived as shown in the following equation.

[0526] [Formula 48]

[0527] qP=Clip3(0,63,qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0528] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be derived as shown in the following equation.

[0529] [Formula 49]

[0530] qP=Clip3(0,63,Max(QpPrimeTsMin,qP)-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[0531] In another embodiment, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, the lowest value of qP may be clipped using the value of QpPrimeTsMin instead of 0 as shown in the following equation.

[0532] [Formula 50]

[0533] The quantization parameter qP can be updated as follows.

[0534] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be derived as shown in the following equation.

[0535] qP=Clip3(0,63,qP-(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0536] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be derived as shown in the following equation.

[0537] qP=Clip3(QpPrimeTsMin,63,qP-cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[0538] In another embodiment, the quantization parameter qP may be updated as follows.

[0539] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be derived as shown in the following equation.

[0540] [Formula 51]

[0541] qP=Clip3(0,63+QpBdOffset,qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0542] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be derived as shown in the following equation.

[0543] [Formula 52]

[0544] qP=Clip3(0,63+QpBdOffset,Max(QpPrimeTsMin,qP)+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[0545] In another embodiment, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, the lowest value of qP may be clipped using the value of QpPrimeTsMin instead of 0 as shown in the following equation.

[0546] [Formula 53]

[0547] The quantization parameter qP can be updated as follows.

[0548] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be derived as shown in the following equation.

[0549] qP=Clip3(0,63+QpBdOffset,qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0550] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be derived as shown in the following equation.

[0551] qP=Clip3(QpPrimeTsMin,63+QpBdOffset,qP+cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[0552] Embodiment 3: Method for allowing ACT when executing chroma BDPCM

[0553] In an embodiment, when BDPCM is applied to a luma component block, ACT may be applied to encode / decode the block. However, when BDPCM is applied to a chroma component block, ACT is limited to not be applied to encode / decode the block.

[0554] Furthermore, even when BDPCM is applied to a chroma component block, ACT is applied to the block, thereby improving encoding efficiency. Fig.37 FIG. 2 shows an embodiment of a syntax configuration in which ACT is applied even when BDPCM is applied to a chrominance component block. Fig.37 As shown, by removing the condition of obtaining the BDPCM syntax element of the chroma component according to the value of cu_act_enabled_flag indicating whether ACT is applied to the current encoding basis, its BDPCM syntax element can be obtained regardless of whether ACT is applied to the chroma component block, and BDCPM encoding can be performed accordingly.

[0555] Embodiment 4: Method of applying ACT even when encoding / decoding is performed using CCLM

[0556] Both CCLM and ACT are designed to remove unnecessary overlap between components. There is some overlap between CCLM and ACT, but even after applying all of them, the overlap between components may not be completely removed. Therefore, by applying CCLM and ACT together, the overlap between components can be removed more.

[0557] The following embodiments describe embodiments in which CCLM and ACT are applied together. When performing decoding, the decoding device may first apply CCLM and then apply ACT. When ACT is applied to both BDPCM and CCLM of the chrominance component, the syntax table that signals this may be as follows: Fig.38 Therefore, if Fig.38 As shown in the syntax table, among the restrictions on signaling syntax elements related to intra_bdpcm_chroma and ccm, if(!cu_act_enabled_flag) that signals syntax elements depending on whether ACT is not applied can be removed from the syntax table.

[0558] Embodiment 5: Method using flexible ACTQp including joint CbCr

[0559] When the ACT mode is applied, the prediction residual can be transformed from one color space (e.g., GBR or YCbCr) to the YCgCo color space. In addition, the residual of the transform basis can be encoded in the YCgCo color space. As an embodiment of the ACT core transform (transform kernel) for color space transform, the following transform kernel can be used as described above.

[0560] [Formula 54]

[0561]

[0562] [Formula 55]

[0563]

[0564] As described in the above formula, C0', C1' and C2' (herein, C0'=Y, C1'=Cg, C2'=Co) transformations are not normalized. For example, the L2 norm does not have a value of 1. For example, the L2 norm of the transformation of each component may have a value of about 0.6 for C0' and C1', and a value of about 0.7 for C2'. Herein, the L2 norm is a value obtained as the square root of the sum of the squares of the corresponding coefficients. For example, C0'=2 / 4*C0+1 / 4*C1+1 / 4*C2 can be calculated. Therefore, the norm of C0' can be calculated as the square root of (2 / 4*2 / 4+1 / 4*1 / 4+1 / 4*1 / 4). Therefore, this can be calculated as the square root of 6 / 16, and can be calculated to have a value of about 0.6.

[0565] When the normalization transform is not applied, the dynamic range of each component is irregular. In addition, this leads to a decrease in the encoding performance of general video compression systems.

[0566] In order to compensate for the dynamic range of the residual signal, a QP offset value for compensating for the dynamic range variation of each transform component is transmitted so that QP adjustment can be performed. For example, this embodiment can be applied to a general QP adjustment control method for ACT transform and joint CbCr.

[0567] The individual color components are not encoded independently but are encoded together, so that the method described above for joint CbCr in embodiment 3 can result in dynamic range changes between the individual color components.

[0568] In the encoding and decoding methods according to the embodiment, the ACT QP offset adjustment may be fixed to -5, which may be applied to Y, Cg, and Co in the same manner.

[0569] In an embodiment, to provide flexible Qp control for each component and joint CbCr, different ACT Qp offsets may be allowed for Y, Cb, Cr and / or joint CbCr. The ACT Qp offset value may be determined based on component index and / or joint CbCr and / or joint CbCr mode.

[0570] To indicate the ACT Qp offset, ppsActQpOffsetY, ppsActQpOffsetCb, and ppsActQpOffsetCr may be used. In addition, ppsActQpOffsetCbCr may be used for the ACTQP offset for joint CbCr mode 2 with CBF, where all Cb and Cr components have non-zero values. These values ​​(e.g., ppsActQpOffsetY, ppsActQpOffsetCb, ppsActQpOffsetCr, and ppsActQpOffsetCbCr) may be predetermined to predetermined values ​​or signaled through the bitstream. The ACT QP offset for the joint CbCr mode may be set in another method or to another value.

[0571] In an embodiment, ACT Qp offsets -5, -5, and -3 may be used for Y, Cb, and Cr, and ACT Qp offset -4 may be used for joint CbCr.

[0572] In another embodiment, ACT Qp offsets -5, -4, and -3 may be used for Y, Cb, and Cr, and ACT Qp offset -3 may be used for joint CbCr mode, where the value of tu_cbf_cb is not zero.

[0573] In yet another embodiment, the ACT QP offset for joint CbCr mode 2 may have its own offset value. For another joint CbCr mode, the ACT QP offset may use the offset of the corresponding component. For example, the quantization parameter qP may be determined as follows. First, when cIdx has a value of 0, qP and the ACT Qp offset may be derived as shown below.

[0574] [Formula 56]

[0575] qP=Qp' Y

[0576] ActQpOffset=ppsActQpOffsetY

[0577] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0578] [Formula 57]

[0579] qP=Qp' CbCr

[0580] ActQpOffset=ppsActQpOffsetCbCr

[0581] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset may be derived as shown below.

[0582] [Formula 58]

[0583] qP=Qp' Cb

[0584] ActQpOffset=ppsActQpOffsetCb

[0585] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0586] [Formula 59]

[0587] qP=Qp' Cr

[0588] ActQpOffset=ppsActQpOffsetCr

[0589] In an embodiment, the quantization parameter qP may be updated as follows.

[0590] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be derived as shown in the following equation.

[0591] [Formula 60]

[0592] qP=Clip3(0,63+QpBdOffset,qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0593] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be derived as shown in the following equation.

[0594] [Formula 61]

[0595] qP=Clip3(QpPrimeTsMin,63+QpBdOffset,qP+cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[0596] In another embodiment, for a joint CbCr mode with tu_cbf_cb!=0 (e.g., in the case of modes 1 and 2), ppsActQpOffsetCb may be used to determine the offset of the joint CbCr. Alternatively, for a joint CbCr mode with tu_cbf_cb == 0 (e.g., in the case of mode 3), ppsActQpOffsetCr may be used to determine the offset of the joint CbCr. For example, the above embodiment may be modified and applied as follows.

[0597] The quantization parameter qP may be updated as follows: First, when cIdx has a value of 0, qP and ACT Qp offset may be derived as shown in the following equation.

[0598] [Formula 62]

[0599] qP=Qp' Y

[0600] ActQpOffset=ppsActQpOffsetY

[0601] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP may be derived as shown below.

[0602] [Formula 63]

[0603] qP=Qp' CbCr

[0604] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset may be derived as shown below.

[0605] [Formula 64]

[0606] qP=Qp' Cb

[0607] ActQpOffset=ppsActQpOffsetCb

[0608] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0609] [Formula 65]

[0610] qP=Qp' Cr

[0611] ActQpOffset=ppsActQpOffsetCr

[0612] When cIdx does not have a value of 0 and TuCResMode[xTbY][yTbY] does not have a value of 0, the ACT Qp offset for the joint CbCr mode may be determined according to the following pseudo code.

[0613] [Formula 66]

[0614] if(TuCResMode[xTbY][yTbY]is euqal to 1or 2)

[0615] ActQpOffset=ppsActQpOffsetCb;

[0616] else

[0617] ActQpOffset=ppsActQpOffsetCr;

[0618] In an embodiment, the quantization parameter qP may be updated as follows: When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be derived as shown in the following equation.

[0619] [Formula 67]

[0620] qP=Clip3(0,63+QpBdOffset,qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0621] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be derived as shown in the following equation.

[0622] [Formula 68]

[0623] qP=Clip3(QpPrimeTsMin,63+QpBdOffset,qP+cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[0624] In another embodiment, regardless of the joint CbCr mode, ppsActQpOffsetY is used when the component index is Y, ppsActQpOffsetCb is used when the component index is Cb, and ppsActQpOffsetCr is used when the component index is Cr, thereby qP can be derived. For example, the quantization parameter qP can be derived as follows.

[0625] First, when cIdx has a value of 0, qP and ACT Qp offset can be derived as shown below.

[0626] [Formula 69]

[0627] qP=Qp' Y

[0628] ActQpOffset=ppsActQpOffsetY

[0629] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0630] [Formula 70]

[0631] qP=Qp' CbCr

[0632] ActQpOffset=(cIdx==1)? ppsActQpOffsetCb:ppsActQpOffsetCr

[0633] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset may be derived as shown below.

[0634] [Formula 71]

[0635] qP=Qp' Cb

[0636] ActQpOffset=ppsActQpOffsetCb

[0637] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0638] [Formula 72]

[0639] qP=Qp' Cr

[0640] ActQpOffset=ppsActQpOffsetCr

[0641] The quantization parameter qP can be updated as follows.

[0642] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP may be derived as shown in the following equation.

[0643] [Formula 73]

[0644] qP=Clip3(0,63+QpBdOffset,qP+(cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0))

[0645] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP may be derived as shown in the following equation.

[0646] [Formula 74]

[0647] qP=Clip3(QpPrimeTsMin,63+QpBdOffset,qP+cu_act_enabled_flag[xTbY][yTbY]?ActQpOffset:0)

[0648] Embodiment 6: Method for signaling ACTQp offset including joint CbCr

[0649] In the following, examples are described of signaling the ACT QP offset through the bitstream to provide more flexibility. The ACT QP offset may be signaled through the SPS, PPS, picture header, slice header, or other types of header sets. The ACT Qp offset for joint CbCr may be signaled separately or may be derived from the ACT QP offsets for Y, Cb, and Cr.

[0650] Without loss of generality, Fig.39 An example of a syntax table for signaling an ACT Qp offset in a PPS is shown. Fig.39 In an embodiment, an ACT Qp offset may be signaled for the joint CbCr. Fig.39 The syntax elements indicated in the syntax table.

[0651] The syntax element pps_act_qp_offsets_present_flag may indicate whether there is a syntax element related to ACT QP offset in PPS. For example, pps_act_qp_offsets_present_flag may indicate whether syntax elements pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4, which will be described later, are signaled as PPS.

[0652] For example, a first value (eg, 0) of pps_act_qp_offsets_present_flag may indicate that pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4 are not signaled through the PPS syntax table.

[0653] A second value (eg, 1) of pps_act_qp_offsets_present_flag may indicate that pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4 are signaled through the PPS syntax table.

[0654] When pps_act_qp_offsets_present_flag is not provided from the bitstream, pps_act_qp_offsets_present_flag may be derived as a first value (e.g., 0). For example, when a flag indicating whether ACT is applied (e.g., sps_act_enabled_flag signaled in SPS) has a first value (e.g., 0) indicating that ACT is not applied, pps_act_qp_offsets_present_flag may be forced to have a first value (e.g., 0).

[0655] When the value of the syntax element cu_act_enabled_flag is the second value (e.g., 1) indicating that ACT is applied to the current coding basis, the syntax elements pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4 may be used to determine the offsets of the quantization parameter values ​​qP applied to the luma, Cb, Cr components, and joint CbCr components, respectively. When the values ​​of pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4 do not exist in the bitstream, the respective values ​​may be set to 0.

[0656] According to the syntax elements, the values ​​of the variables PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr and PpsActQpOffsetCbCr can be determined as shown in the following formula.

[0657] [Formula 75]

[0658] PpsActQpOffsetY=pps_act_y_qp_offset_plusX1-X1

[0659] PpsActQpOffsetCb=pps_act_cb_qp_offset_plusX2-X2

[0660] PpsActQpOffsetCr=pps_act_cr_qp_offset_plusX3-X3

[0661] PpsActQpOffsetCbCr=pps_act_cbcr_qp_offset_plusX4-X4

[0662] Herein, X1, X2, X3, and X4 may indicate predetermined constant values. These may be the same value or different values, or only some may have the same value.

[0663] In an embodiment, for bitstream matching, the values ​​of PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, and PpsActQpOffsetCbCr may be restricted to have values ​​within the range of -12 to 12.

[0664] According to the determination of the variables, the quantization parameter qP may be determined as follows: First, when cIdx has a value of 0, qP and ACT Qp offset may be derived as shown in the following equation.

[0665] [Formula 76]

[0666] qP=Qp' Y

[0667] ActQpOffset=PpsActQpOffsetY

[0668] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0669] [Formula 77]

[0670] qP=Qp' CbCr

[0671] ActQpOffset=PpsActQpOffsetCbCr

[0672] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset may be derived as shown below.

[0673] [Formula 78]

[0674] qP=Qp' Cb

[0675] ActQpOffset=PpsActQpOffsetCb

[0676] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0677] [Formula 79]

[0678] qP=Qp' Cr

[0679] ActQpOffset=PpsActQpOffsetCr

[0680] In another embodiment of signaling ACT Qp offsets, multiple ACT QP offsets may be signaled for different joint CbCr modes identified as Mode-A and Mode-B.

[0681] The joint CbCr mode A may refer to a joint CbCr mode including tu_cbf_cb with a non-zero value, such as Mode 1 and Mode 2 of Table 2 above. The joint CbCr mode B may refer to a joint CbCr mode including tu_cbf_cb with a 0 value, such as Mode 3 of Table 2 above. The corresponding changed syntax is represented by Fig.40 In. Will describe Fig.40 The syntax elements indicated in the syntax table.

[0682] When the value of the syntax element cu_act_enabled_flag is the second value (e.g., 1) indicating that ACT is applied for the current coding basis, the syntax elements pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, pps_act_cbcr_qp_offset_modeA_plusX4, and pps_act_cbcr_qp_offset_modeB_plusX5 may be used to determine the offsets of the quantization parameter values ​​qP applied to the luma, Cb, Cr components, and joint CbCr components, respectively. When the values ​​of pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, pps_act_cbcr_qp_offset_modeA_plusX4, and pps_act_cbcr_qp_offset_modeB_plusX5 do not exist in the bitstream, each value may be set to 0.

[0683] According to the syntax elements, the values ​​of the variables PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, PpsActQpOffsetCbCrModeA and PpsActQpOffsetCbCrModeB can be determined as shown in the following formula.

[0684] [Formula 80]

[0685] PpsActQpOffsetY=pps_act_y_qp_offset_plusX1-X1

[0686] PpsActQpOffsetCb=pps_act_cb_qp_offset_plusX2-X2

[0687] PpsActQpOffsetCr=pps_act_cr_qp_offset_plusX3-X3

[0688] PpsActQpOffsetCbCrModeA=pps_act_cbcr_qp_offset_modeA_plusX4-X4

[0689] PpsActQpOffsetCbCrModeB=pps_act_cbcr_qp_offset_modeB_plusX5-X5

[0690] Herein, X1, X2, X3, X4, and X5 may indicate predetermined constant values. These may be the same value or different values, or only some may have the same value. In an embodiment, for bitstream matching, the values ​​of PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, PpsActQpOffsetCbCrModeA, and PpsActQpOffsetCbCrModeB may be restricted to have values ​​in the range of -12 to 12.

[0691] According to the determination of the variables, the quantization parameter qP may be determined as follows: First, when cIdx has a value of 0, qP and ACT Qp offset may be derived as shown in the following equation.

[0692] [Formula 81]

[0693] qP=Qp' Y

[0694] ActQpOffset=PpsActQpOffsetY

[0695] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP may be derived as shown below.

[0696] [Formula 82]

[0697] qP=Qp' CbCr

[0698] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset may be derived as shown below.

[0699] [Formula 83]

[0700] qP=Qp' Cb

[0701] ActQpOffset=PpsActQpOffsetCb

[0702] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0703] [Formula 84]

[0704] qP=Qp' Cr

[0705] ActQpOffset=PpsActQpOffsetCr

[0706] In addition, when cIdx does not have a value of 0 and TuCResMode[xTbY][yTbY] does not have a value of 0, the ACT Qp offset may be derived as shown in the following equation.

[0707] [Formula 85]

[0708] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])? PpsActQpOffsetCbCrModeA:PpsActQpOffsetCbCrMod eB

[0709] Furthermore, in another embodiment, when TuCResMode[xTbY][yTbY] has a value of 2, ActQpOffset may be derived as shown in the following equation.

[0710] [Formula 86]

[0711] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])? (PPsQpOffsetCbCrModeA+slice_act_CbCr_qp_offset_ModeA):(PPsQpOffsetCbCrModeB+slice_act_CbCr_qp_offset_ModeB)

[0712] In another embodiment of signaling the ACT Qp offset, it can be as follows Fig.41 Only the ACT QP offsets for Y, Cb, and Cr are signaled in the syntax table of . The ACT QP offset for joint CbCr can be derived from PpsActQpOffsetY, PpsActQpOffsetCb, and / or PpsActQpOffsetCr.

[0713] In an embodiment, the ACT Qp offset of CbCr may be set to the value of PpsActQpOffsetCb. In another embodiment, the ACT Qp offset of CbCr may be set to the same value as PpsActQpOffsetCb in the case of joint CbCr mode where tu_cbf_cb has a non-zero value, or may be set to the same value as PpsActQpOffsetCr in the case of joint CbCr mode where tu_cbf_cb has a value of 0. Alternatively, it may be set vice versa.

[0714] Fig.41 FIG. 4 is a diagram showing another embodiment of a syntax table for signaling an ACT Qp offset in a PPS. Fig.41 The quantization parameter qP can be determined as follows: First, when cIdx has a value of 0, qP and ACT Qp offset can be derived as shown in the following formula.

[0715] [Formula 87]

[0716] qP=Qp' Y

[0717] ActQpOffset=PpsActQpOffsetY

[0718] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0719] [Formula 88]

[0720] qP=Qp' CbCr

[0721] ActQpOffset=(cIdx==1)? PpsActQpOffsetCb:PpsActQpOffsetCr

[0722] Furthermore, in another embodiment, the value of ActQpOffset may be determined as follows.

[0723] [Formula 89]

[0724] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])? PpsActQpOffsetCb:PpsActQpOffsetCr

[0725] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset may be derived as shown below.

[0726] [Formula 90]

[0727] qP=Qp' Cb

[0728] ActQpOffset=PpsActQpOffsetCb

[0729] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0730] [Formula 91]

[0731] qP=Qp' Cr

[0732] ActQpOffset=PpsActQpOffsetCr

[0733] Embodiment 7: Signaling ACT Qp offset at multiple levels

[0734] In an embodiment, the ACT QP offset may be signaled at multiple levels. In addition to signaling the ACT QP offset at one level (such as PPS) as in Embodiment 6 above, the ACT QP offset may be signaled at a lower level (e.g., a slice header, a picture header, or other types of headers suitable for Qp control).

[0735] Hereinafter, two embodiments will be described. Fig.42 and Fig.43 An example of signaling an ACT QP offset through a slice header and a picture header is shown. In this way, the ACT QP offset can be signaled at multiple levels.

[0736] Hereinafter, the Fig.42 and Fig.43 The syntax element pps_slice_act_qp_offsets_present_flag may indicate whether the syntax elements slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset (to be described later) are present in the slice header.

[0737] For example, a first value (eg, 0) of pps_slice_act_qp_offsets_present_flag may indicate that slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset are not present in the slice header.

[0738] For example, a second value (eg, 1) of pps_slice_act_qp_offsets_present_flag may indicate that slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset are present in the slice header.

[0739] The syntax elements slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset may indicate offsets of quantization parameter values ​​qP for luma, Cb, Cr components, and joint CbCr components, respectively. The values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset may be restricted to have values ​​in the range of -12 to 12. When the values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset do not exist in the bitstream, the respective values ​​may be set to 0. The values ​​of PpsActQpOffsetY+slice_act_y_qp_offset, PpsActQpOffsetCb+slice_act_cb_qp_offset, PpsActQpOffsetCr+slice_act_cr_qp_offset, and PpsActQpOffsetCbCr+slice_act_cbcr_qp_offset may be restricted to also have values ​​within the range of -12 to 12.

[0740] Various modified implementations of signaling the ACT QP offset of joint CbCr at the PPS level may be applied. For example, one QP offset may be signaled for joint CbCr, multiple ACT Qp offsets may be signaled for different modes of joint CbCr, or there is no need to signal the ACT Qp offset for joint CbCr, and a method of deriving it by using the ACTQp offsets of the modes of Y, Cb and Cr and / or joint CbCr may be applied when signaling through a slice header is performed.

[0741] Fig.44 and Fig.45Two modified embodiments are shown. Fig.44 An implementation in which the ACT Qp offset is signaled in the slice header is shown. Fig.45 Another embodiment of signaling the ACT Qp offset in the slice header is shown. Fig.45 In the embodiment, only the ACT Qp offsets of Y, Cb, and Cr may be signaled, and the slice-level ACT QP offset of the joint CbCr may be derived from slice_act_y_qp_offset, slice_act_cb_qp_offset, and / or slice_act_cr_qp_offset. This may be determined based on the mode type of the joint CbCr. In an embodiment, the slice-level ACT Qp offset of CbCr may be set to the same value as slice_act_cb_qp_offset. In another embodiment, in the case of a joint CbCr mode including tu_cbf_cb with a non-zero value, the slice-level ACT Qp offset of the joint CbCr may be set to the same value as slice_act_cb_qp_offset. In addition, in the case of a joint CbCr mode including tu_cbf_cb with a value of 0, the slice-level ACT Qp offset of the joint CbCr may be set to the same value as slice_act_cr_qp_offset.

[0742] Furthermore, in another embodiment, the syntax elements may be signaled in a slice header or a picture header. To achieve this, encoding / decoding may be performed as follows.

[0743] - A flag pps_picture_slice_act_qp_offsets_present_flag indicating whether the ACT Qp offset is present in the picture header or in the slice header may be signaled in the PPS.

[0744] -When ACT is applicable and the value of pps_picture_slice_act_qp_offsets_present_flag is a second value (e.g., 1), a flag pic_act_qp_offsets_present_flag indicating whether there is an ACT Qp offset in the picture header is signaled in the picture header. Herein, the second value (e.g., 1) of pic_act_qp_offsets_present_flag may indicate that the ACT Qp offsets of all slices of the picture corresponding to the picture header are provided in the picture header.

[0745] - A first value (e.g., 0) of pic_act_qp_offsets_present_flag may indicate that the ACT Qp offsets of all slices of the picture corresponding to the picture header are not provided in the picture header. For example, when ACT is applicable and the value of pps_picture_slice_act_qp_offsets_present_flag is a second value (e.g., 1) and the value of pic_act_qp_offsets_present_flag is a first value (e.g., 0), the ACT Qp offsets of the slice may be provided in the slice header.

[0746] Fig.46 is a view of a syntax table of a PPS showing signaling of pps_pic_slice_act_qp_offsets_present_flag. The syntax element pps_pic_slice_act_qp_offsets_present_flag may indicate whether the ACT Qp offset is provided in a picture header and / or a slice header. For example, a first value of pps_pic_slice_act_qp_offsets_present_flag (e.g., 0) may indicate that the ACT Qp offset is not provided in the picture header and the slice header. A second value of pps_pic_slice_act_qp_offsets_present_flag (e.g., 1) may indicate that the ACT Qp offset is provided in the picture header or the slice header. When pps_pic_slice_act_qp_offsets_present_flag is not provided in the bitstream, the value of pps_pic_slice_act_qp_offsets_present_flag may be determined to be a first value (e.g., 0).

[0747] Fig.47 is a view of a syntax table showing a picture header that signals an ACT Qp offset. The syntax element pic_act_qp_offsets_present_flag may indicate whether the ACT Qp offset is provided in the picture header. A first value of pic_act_qp_offsets_present_flag (e.g., 0) may indicate that the ACT Qp offset is not provided in the picture header but in the slice header. A second value of pic_act_qp_offsets_present_flag (e.g., 1) may indicate that the ACT Qp offset is provided in the picture header. When the value of pic_act_qp_offsets_present_flag is not provided in the bitstream, the value may be determined to be 0.

[0748] Fig.48 is a diagram showing a syntax table of a slice header that signals an ACT Qp offset. Fig.48 In the syntax table of , the syntax elements slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset may indicate the offset of the quantization parameter value qP of the luma, Cb, and Cr components. The values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset may have a value in the range of -12 to 12. In addition, the values ​​of PpsActQpOffsetY+slice_act_y_qp_offset, PpsActQpOffsetCb+slice_act_cb_qp_offset, and PpsActQpOffsetCr+slice_act_cr_qp_offset may be restricted to also have a value range of -12 to 12.

[0749] Furthermore, in a case where the values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset are not provided in a bitstream, when the value of pps_pic_slice_act_qp_offsets_present_flag is the first value (eg, 0), the values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, and slice_act_cr_qp_offset may be determined to be 0. Alternatively, when the value of pps_pic_slice_act_qp_offsets_present_flag is the second value (eg, 1), the values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, and slice_act_cr_qp_offset may be determined to be the same as pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset, respectively.

[0750] Furthermore, when an ACT Qp offset exists in both a slice header and a picture header, a final offset value for deriving a qP value may be determined as a value obtained by adding an offset value signaled in a PPS to an offset value signaled in a slice header or a picture header.

[0751] More specifically, in an embodiment, the quantization parameter qP may be determined as follows: First, when cIdx has a value of 0, qP and ACT Qp offset may be derived as shown in the following equation.

[0752] [Formula 92]

[0753] qP=Qp' Y

[0754] ActQpOffset=PPsQpOffsetY+slice_act_y_qp_offset

[0755] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0756] [Formula 93]

[0757] qP=Qp' CbCr

[0758] ActQpOffset=PPsQpOffsetCbCr+slice_act_CbCr_qp_offset

[0759] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset may be derived as shown below.

[0760] [Formula 94]

[0761] qP=Qp' Cb

[0762] ActQpOffset=PpsActQpOffsetCb+slice_act_Cb_qp_offset

[0763] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0764] [Formula 95]

[0765] qP=Qp' Cr

[0766] ActQpOffset=PpsActQpOffsetCr+slice_act_Cr_qp_offset

[0767] In another embodiment, when multiple ACT Qp offsets of a joint CbCr are signaled, the ActQpOffset of the joint CbCr may be determined as follows.

[0768] First, when cIdx has a value of 0, qP and ACT Qp offset can be derived as shown below.

[0769] [Formula 96]

[0770] qP=Qp' Y

[0771] ActQpOffset=PPsQpOffsetY+slice_act_y_qp_offset

[0772] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP may be derived as shown below.

[0773] [Formula 97]

[0774] qP=Qp' CbCr

[0775] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset may be derived as shown below.

[0776] [Formula 98]

[0777] qP=Qp' Cb

[0778] ActQpOffset=PpsActQpOffsetCb+slice_act_Cb_qp_offset

[0779] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0780] [Formula 99]

[0781] qP=Qp' Cr

[0782] ActQpOffset=PpsActQpOffsetCr+slice_act_Cr_qp_offset

[0783] In addition, when cIdx does not have a value of 0 and TuCResMode[xTbY][yTbY] does not have a value of 0, the ACT Qp offset may be derived as shown in the following equation.

[0784] [Formula 100]

[0785] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])? (PPsQpOffsetCbCrModeA+slice_act_CbCr_qp_offset_ModeA):(PPsQpOffsetCbCrModeB+slice_act_CbCr_qp_offset_ModeB)

[0786] In yet another embodiment, when the ACT Qp offset of the joint CbCr is not provided, ActQpOffset and qP of the Y, Cb and / or Cr components are determined, and the ActQpOffset of the joint CbCr may be determined using the ACT Qp offset of the Y, Cb and / or Cr components as follows. For example, in the above embodiment, when TuCResMode[xTbY][yTbY] related to Equation 97 has a value of 2, the calculation step of qP may be changed and performed as follows.

[0787] “Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offsets may be derived as shown below.

[0788] [Formula 101]

[0789] qP=Qp' CbCr

[0790] ActQpOffset=(cIdx==1])? (PPsQpOffsetCb+slice_act_Cb_qp_offset):(PPsQpOffsetCr+slice_act_Cr_qp_offset)”

[0791] In addition, in another embodiment, the value of ActQpOffset may be determined as follows.

[0792] [Formula 102]

[0793] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])? (PPsQpOffsetCb+slice_act_Cb_qp_offset):(PPsQpOffset Cr+slice_act_Cr_qp_offset)

[0794] Embodiment 8: Method for signaling multiple ACTQp offset sets

[0795] In this embodiment, a method of using a list of ACT Qp offsets will be described. To this end, the following process may be performed.

[0796] a) Multiple sets of ACT Qp offsets may be signaled in the form of a list within a parameter set (e.g., SPS or PPS). Each set in the list may include ACT Qp offsets for Y, Cb, Cr, and joint CbCr components. For simplicity, the list of ACTQp offsets may be signaled in the same parameter set as the parameter set used to signal the list of chroma Qp offsets.

[0797] b) The number of sets of ACT Qp offsets in the list may be the same as the number of chroma Qp offset sets signaled in the PPS.

[0798] c) As the ACT Qp offset for deriving the qP of each coding basis, an ACT Qp offset belonging to a list with an index of the chroma Qp offset of the coding basis (eg, cu_chroma_qp_offset_idx) may be used.

[0799] d) As an alternative to b) and c), the following operations may be performed.

[0800] - The number of sets of ACT Qp offsets in the list may be signaled. The number of sets of ACT Qp offsets in the list may be different from the number of sets of chroma Qp offsets.

[0801] - When ACT is applicable, an index indicating the index of the ACT Qp offset used for encoding basis may be signaled.

[0802] Without departing from the above concepts, the syntax for signaling a list of ACT Qp offsets may be as follows: Fig.49 For example, when cu_act_enabled_flag has a value of 1, pps_act_y_qp_offset, pps_act_cb_qp_offset, pps_act_cr_qp_offset, and pps_act_cbcr_qp_offset may be used to determine offsets of quantization parameter values ​​qP to be applied to luma, Cb, Cr components, and joint CbCr, respectively.

[0803] When the values ​​of pps_act_y_qp_offset, pps_act_cb_qp_offset, pps_act_cr_qp_offset, and pps_act_cbcr_qp_offset do not exist, the respective values ​​may be derived to be 0.

[0804] When the value of cu_act_enabled_flag is the second value (e.g., 1) and the value of cu_chroma_qp_offset_flag is the second value (e.g., 1), act_y_qp_offset_list[i], act_cb_qp_offset_list[i], act_cr_qp_offset_list[i], and act_cbcr_qp_offset_list[i] may be used to determine the offsets of the quantization parameter values ​​qP applied to the luma, Cb, Cr components, and joint CbCr components, respectively. When there are no values ​​of act_y_qp_offset_list[i], act_cb_qp_offset_list[i], act_cr_qp_offset_list[i], and act_cbcr_qp_offset_list[i], the respective values ​​may be derived to be 0.

[0805] In this embodiment, the quantization parameter qP may be determined as follows: First, when cIdx has a value of 0, qP and ACT Qp offset may be derived as shown in the following equation.

[0806] [Formula 103]

[0807] qP=Qp' Y

[0808] ActQpOffset=pps_act_y_qp_offset+(cu_chroma_qp_offset_flag)? act_y_qp_offset_list[cu_chroma_qp_offset_idx]:0+slice_act_y_qp_offset

[0809] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0810] [Formula 104]

[0811] qP=Qp' C b C r

[0812] ActQpOffset=pps_act_cbcr_qp_offset+(cu_chroma_qp_offset_flag)? act_cbcr_qp_offset_list[cu_chroma_qp_offset_idx]:0+slice_act_cbcr_qp_offset

[0813] Alternatively, when cIdx has a value of 1, qP and ACT Qp offset may be derived as shown below.

[0814] [Formula 105]

[0815] qP=Qp' C b

[0816] ActQpOffset=pps_act_cb_qp_offset+(cu_chroma_qp_offset_flag)? act_cb_qp_offset_list[cu_chroma_qp_offset_idx]:0+slice_act_cb_qp_offset

[0817] Alternatively, when cIdx has a value of 2, qP and ACT Qp offset may be derived as shown below.

[0818] [Equation 106]

[0819] qP=Qp' C r

[0820] ActQpOffset=pps_act_cr_qp_offset+(cu_chroma_qp_offset_flag)? act_cr_qp_offset_list[cu_chroma_qp_offset_idx]:0+slice_act_cr_qp_offset

[0821] Embodiment 9: ACT color space conversion method to be applied to both lossless coding and lossy coding

[0822] The transformation between color spaces based on the above matrices for forward transformation and backward transformation can be organized as follows.

[0823] [Table 3]

[0824]

[0825] Since some values ​​are lost in the processing of Co and Cg, the transformation cannot achieve reconstruction to the original state. For example, when the sample values ​​of the RGB color space are converted to the sample values ​​of the YCgCo color space and the resulting values ​​are inversely converted back to the sample values ​​of the RGB color space, the values ​​of the original samples are not completely reconstructed. Therefore, the transformation according to Table 3 cannot be used for lossless coding. It is necessary to improve the color space transformation algorithm so that no loss of sample values ​​occurs after the color space transformation even when lossless coding is applied. Implementation 9 and Implementation 10 describe color space transformation algorithms that can be applied to lossless coding as well as lossy coding.

[0826] In the following embodiments, a method for performing ACT by using a color space transform that is reversible to the original state is described, which is applicable to lossy encoding as well as lossless encoding. The reversible color space transform can be applied to the above-mentioned encoding and decoding methods. The ACT Qp offset can also be adjusted for the following color space transform. The color space transform according to the embodiment can be performed as shown in the following formula. For example, the forward transform from the GBR color space to the YCgCo color space can be performed according to the following formula.

[0827] [Equation 107]

[0828] Co=RB;

[0829] t=B+(Co>>1);

[0830] Cg=Gt;

[0831] Y=t+(Cg>>1);

[0832] In addition, the backward transformation from the YCgCo color space to the GBR color space can be performed according to the following equation.

[0833] [Equation 108]

[0834] t=Y-(Cg>>1)

[0835] G=Cg+t

[0836] B=t-(Co>>1)

[0837] R=Co+B

[0838] The transformation between the YCgCo color space and the RGB color space according to the above formula is reversible to the original state. That is, the color space transformation according to the formula supports perfect reconstruction. For example, the sample value is maintained even if the backward transformation is performed after the forward transformation. Therefore, the color space transformation according to the formula can be called a reversible YCgCo-R color transformation. Herein, R can represent reversible, which means that reconstruction to the original state is achieved. A YCgCo-R transformation can be provided by increasing the bit depth of Cg and Co by 1 compared to the existing transformation. If this condition is met, other types of reversible transformations can be used similar to the above transformation.

[0839] Since the transform as shown above has a different norm value from the above transform, the ACT Qp offsets of Y, Cg, and Co can be adjusted to compensate for the dynamic range change caused by the color space transform.

[0840] It has been described that when the above-mentioned transformation is applied, the QCT Qp offset according to the embodiment may have the value (-5, -5, -5) for Y, Cg, and Co. However, when the reversible transformation according to the present embodiment is applied, another value other than (-5, -5, -5) may be specified for the QCT Qp offset according to the embodiment. For example, the values ​​(-5, 1, 3) of Y, Cg, and Co may be used for the QCT Qp offset according to the embodiment.

[0841] In another embodiment, the ACT QP offset may be signaled via a bitstream as in Embodiment 6 or Embodiment 7 above.

[0842] For example, when the above YCgCo-R transform is used with an ACT QP offset of (-5, 1, 3), as shown in the following figure, it is observed that there is no coding loss in the lossy coding environment (e.g., QP 22, 27, 32, 37). In addition, it is observed that when ACT is applied, a further 5% coding performance is obtained in achieving lossless coding.

[0843] [Table 4]

[0844] sequence Y U V RGB, TGM 1080p 0.0％ 0.2％ 0.1％ RGB, TGM 720p 0.2％ -0.1％ 0.1％ RGB, Animation -0.1％ -0.1％ 0.0％ RGB, mixed content -0.1％ 0.0％ -0.1％ RGB, camera capture content -0.3％ 0.2％ -0.3％ Total(RGB) 0.0％ 0.0％ 0.0％

[0845] The VVC specification including the integrated ACT matrix can be described as follows.

[0846] [Table 5]

[0847]

[0848] For example, the residual sample array r of size (nTbW)×(nTbH) Y 、r Cb and r Cr It can be updated as follows.

[0849] [Equation 109]

[0850] tmp=r Y [x][y]-(r Cb [x][y]>>1)

[0851] r Y [x][y]=tmp+r Cb [x][y]

[0852] r Cb [x][y]=tmp-(r Cr [x][y]>>1)

[0853] r Cr [x][y]=r Cb [x][y]+r Cr[x][y]

[0854] Implementation 10: ACT execution method for performing multiple color transformations based on explicit signaling

[0855] In this embodiment, at least one color transform may be performed by the ACT. Which color transform to perform may be determined by a flag signaled in the bitstream. The flag may be signaled at multiple levels or identifiable granularities such as SPS, PPS, picture header, and slice.

[0856] In an embodiment, a predetermined flag may be signaled to indicate which ACT to apply. For example, when the flag has a value of 1, an ACT based on a reversible color transformation may be applied. When the flag has a value of 0, an ACT based on an irreversible color transformation may be applied.

[0857] In another embodiment, a predetermined flag of the ACT may be signaled to indicate which color transform to use. Fig.50 An example of the syntax signaled in SPS is shown in FIG. Fig.50 The syntax element sps_act_reversible_conversion may indicate whether to use a transformation that is irreversible to the original state. The first value of sps_act_reversible_conversion (e.g., 0) may indicate that ACT uses a transformation that is irreversible to the original state. The second value of sps_act_reversible_conversion (e.g., 1) may indicate that ACT uses a transformation that is reversible to the original state.

[0858] Therefore, the variable lossyCoding indicating whether to perform lossy coding can be set as follows.

[0859] [Formula 110]

[0860] lossyCoding=(!sps_act_reversible_conversion)

[0861] By using the lossyCoding flag, a pseudo code for the decoding apparatus to perform backward conversion from YCgCo to GBR in the decoding process can be expressed as follows.

[0862] [Formula 111]

[0863] If(sps_act_reversible_conversion==1)

[0864] {

[0865] / / YCgCo-R reversible conversion

[0866] t=Y-(Cg>>1)

[0867] G=Cg+t

[0868] B=t-(Co>>1)

[0869] R=Co+B

[0870] }

[0871] else{

[0872] t=Y-Cg

[0873] G=Y+Cg

[0874] B=t-Co

[0875] R=t+Co

[0876] }

[0877] Therefore, the VVC specification shown in Table 5 of Embodiment 9 may be modified as shown in the following table.

[0878] [Table 6]

[0879]

[0880] According to the above table, the residual update process using color space transformation can use the following parameters as input to this process.

[0881] - variable nTbW, which indicates the block width

[0882] - variable nTbH, which indicates the block height

[0883] - (nTbW)×(nTbH) size array r of luma residual samples Y , which consists of element r Y [x][y],

[0884] - (nTbW)×(nTbH) size array r of chroma residual samples Cb , which consists of element r Cb [x][y],

[0885] - (nTbW)×(nTbH) size array r of chroma residual samples Cr , which consists of element r Cr [x][y],

[0886] The output of this process is as follows.

[0887] - An update array r of size (nTbW) × (nTbH) of luma residual samples Y ,

[0888] - An update array r of size (nTbW) × (nTbH) of chroma residual samples Cb ,

[0889] - An update array r of size (nTbW) × (nTbH) of chroma residual samples Cr ,

[0890] By executing this process, the residual sample array r of size (nTbW)×(nTbH) Y 、r Cb and r Cr It can be updated as follows.

[0891] First, when the value of sps_act_reversible_conversion is the second value (eg, 1), the residual sample array r of size (nTbW)×(nTbH) Y 、r Cb and r Cr It can be updated as shown below.

[0892] [Formula 112]

[0893] tmp=r Y [x][y]-(r Cb [x][y]>>1))

[0894] r Y [x][y]=tmp+r Cb [x][y])

[0895] r Cb [x][y]=tmp-(r Cr [x][y]>>1))

[0896] r Cr [x][y]=r Cb [x][y]+r Cr [x][y]

[0897] Otherwise (eg, when the value of sps_act_reversible_conversion is the first value (eg, 0)), the residual sample array r of size (nTbW)×(nTbH) is Y 、r Cb and r Cr It can be updated as shown below.

[0898] [Formula 113]

[0899] tmp=r Y [x][y]-r Cb [x][y]

[0900] rY [x][y]=r Y [x][y]+r Cb [x][y]

[0901] r Cb [x][y]=tmp-r Cr [x][y]

[0902] r Cr [x][y]=tmp+r Cr [x][y]

[0903] The YCgCo backward transform and the YCgCo-R backward transform have some similarities. In the transform reversible to the original state, when Cg and Co are replaced by Cg'=Cg<<1 and Co'=Co<<1, this can be operated as a lossy backward transform. The following formula shows its implementation.

[0904] [Formula 114]

[0905] t=Y-(Cg'>>1)=Y-Cg

[0906] G=Cg'+t=Y+Cg

[0907] B=t-(Co'>>1)=t-Co=Y-Cg-Co

[0908] R=Co'+B=t+Co=Y-Cg+Co

[0909] Therefore, in an alternative embodiment, instead of maintaining two color transforms, only a transform that is reversible to the original state can be used. In the case of lossy encoding, the Cg and Co components can be scaled by a factor of 1 / 2 in the operation of the encoding device and by a factor of two in the operation of the decoding device. This enables the use of one integrated transform even when supporting lossy and lossless cases. In addition, there is the additional advantage that the bit depth does not change even when lossy encoding is being performed.

[0910] [Table 7]

[0911]

[0912] In an implementation, the Fig.51 The syntax uses a flag (e.g., actShiftFlag) that indicates which ACT shift to use. Fig.51In the syntax table of sps_act_shift_flag, the syntax element sps_act_shift_flag may indicate whether to apply a color component shift while applying ACT. For example, a first value of sps_act_shift_flag (e.g., 0) may indicate that a color component shift is not applied while applying ACT. A second value of sps_act_shift_flag (e.g., 1) may indicate that a color component shift is applied while applying ACT. The variable actShiftFlag may be set to the value of sps_act_shift_flag. The pseudocode for implementing a backward transform from YCgCo to GBR in a decoding device may be written as follows using actShiftFlag.

[0913] [Table 8]

[0914]

[0915] Embodiment 11: ACT execution method for performing multiple color transformations using transformation type derivation

[0916] In an embodiment, at least one color transform may be used when performing an ACT. In addition, which color transform type to use may be derived based on other information in the bitstream.

[0917] In an embodiment, two types of ACT transformations may be available, including an ACT transformation that is reversible to the original state and an ACT transformation that is irreversible to the original state. The ACT transformation type may be derived from the transformation type. For example, as indicated by the variable tuIsTransformSkip, when the transformation type is transform skip, an ACT transformation that is reversible to the original state may be used. Otherwise (e.g., when the transformation type is not transform skip), an ACT transformation that is irreversible to the original state may be used. Two types of pseudocodes may be used.

[0918] [Table 9]

[0919]

[0920] [Table 10]

[0921]

[0922] In another embodiment, the ACT transform type may be determined based on the QP value. When the Qp value is less than or equal to a predetermined threshold (e.g., QpPrimeTsMin), an ACT transform that is reversible to the original state may be used. Otherwise (e.g., when the Qp value exceeds the predetermined threshold), an irreversible ACT transform may be used.

[0923] Embodiment 12: QP derivation method using ACTQP offset

[0924] This embodiment is related to the above-mentioned embodiment 1 and embodiment 2. In embodiment 1 and embodiment 2, the derivation of Qp' has been described. Y , Qp' CbCr , Qp' Cb and Qp' Cr The method described in Embodiment 1 and Embodiment 2 corrects the derived Qp value by using the ACT QP offset, and applies necessary clipping techniques so that the corrected QP value scaled by the transform coefficient does not exceed the valid range.

[0925] This embodiment describes the method for deriving Qp' Y , Qp' CbCr , Qp' Cb and Qp' Cr A method of including an ACTQP offset in the QP derivation process of the transform coefficient scaling process. The QP derivation process already includes a predetermined clipping step so that the derived QP value does not exceed the valid range. Therefore, including the ACT QP offset in the QP derivation process can avoid additional clipping steps, simplify the overall QP derivation steps of the transform coefficient scaling process, and ensure that the final QP does not exceed the valid range.

[0926] As described in the above-mentioned embodiments, the ACT QP offset may be pre-specified as a constant or may be signaled through a bitstream. Without loss of consistency, the ACT QP offsets of Y, Cb, Cr, and CbCr may be described as ppsActQpOffsetY, ppsActQpOffsetCb, ppsActQpOffsetCr, and ppsActQpOffsetCbCr in the description described later. ppsActQpOffsetY, ppsActQpOffsetCb, ppsActQpOffsetCr, and ppsActQpOffsetCbCr may be constants or variables having values ​​ranging from -M to N. Herein, in an embodiment, each of M and N may be set to 12 in the case of lossy coding and may be set to 0 in the case of lossless coding. In addition, at least one ACT QP offset may be derived from another ACT QP offset value. For example, ppsActQpOffsetCbCr may be set to the same value as ppsActQpOffsetCb or ppsActQpOffsetCr based on the joint-CbCr mode.

[0927] The decoding process of QP derivation using the ACT QP offset may be performed as follows: First, in the case of a quantization parameter derivation process, the following parameters may be used for the present process.

[0928] - Luma coordinates (xCb, yCb), which indicate the relative coordinates of the top left luma sample of the current coding block relative to the top left luma sample of the current picture

[0929] - The variable cbWidth indicates the width of the current coding block on a per luma sample basis

[0930] - The variable cbHeight, which indicates the height of the current coding block on a per luma sample basis

[0931] - variable treeType, which indicates whether a single tree (SINGLE_TREE) or a dual tree is used for partitioning the current coding tree node, and indicates whether the dual tree is a luma component dual tree (DAUL_TREE_LUMA) or a chroma component dual tree (DAUL_TREE_CHROMA) when a dual tree is used

[0932] In this quantization parameter derivation process, the brightness quantization parameter Qp' can be derived. Y and the chromaticity quantization parameter Qp' Cb , Qp' Cr and Qp' CbCr .

[0933] After that, the variable Qp Y It can be derived as shown below.

[0934] [Formula 115]

[0935] Q Y =((qP Y_PRED +CuQpDeltaVal+64+2*QpBdOffset)%(64+QpBdOffset))-QpBdOffset

[0936] Luma quantization parameter Qp' Y It can be derived as shown below.

[0937] [Formula 116]

[0938] actQpOffsetY=cu_act_enabled_flag[xCb][yCb]? ppsActQpOffsetY:0

[0939] Qp' Y =Qp Y +QpBdOffset+actQpOffsetY

[0940] Qp' Y =Clip3(0,63+QpBdOffset,Qp' Y )

[0941] When the value of the variable ChromaArrayType indicating the type of the chroma array is not the first value (eg, 0) and treeType is SINGLE_TREE or DUAL_TREE_CHROMA, the following processing may be performed.

[0942] -When the value of treeType is DUAL_TREE_CHROMA, the variable Qp Y The value of may be set to the luma quantization parameter Qp corresponding to the luma coding basis covering the luma sample position (xCb+cbWidth / 2, yCb+cbHeight / 2) Y The value of is the same as the value of .

[0943] -Variable qP Cb ,qP Cr and qP CbCr It can be derived as shown below.

[0944] [Formula 117]

[0945] Q Chroma =Clip3(-QpBdOffset, 63, Qp Y ) (1118)

[0946] Q Cb =ChromaQpTable[0][qP Chroma ] (1119)

[0947] Q Cr =ChromaQpTable[1][qP Chroma ] (1120)

[0948] Q CbCr =ChromaQpTable[2][qP Chroma ] (1121)

[0949] - Chroma quantization parameter Qp' of Cb and Cr components Cb and Qp' Cr And the chromaticity quantization parameter Qp' of the joint Cb-Cr coding CbCr It can be derived as shown below.

[0950] [Formula 118]

[0951] actQpOffsetCb=cu_act_enabled_flag[xCb][yCb]? ppsActQpOffsetCb:0

[0952] actQpOffsetCr=cu_act_enabled_flag[xCb][yCb]? ppsActQpOffsetCr:0

[0953] actQpOffsetCbCr=cu_act_enabled_flag[xCb][yCb]? ppsActQpOffsetCbCr:0

[0954] Qp' Cb =Clip3(-QpBdOffset,63,

[0955] Q Cb +pps_cb_qp_offset+slice_cb_qp_offset+CuQpOffset Cb +actQpOffsetCb)+QpBdOffset

[0956] Qp' Cr =Clip3(-QpBdOffset,63,

[0957] Q Cr +pps_cr_qp_offset+slice_cr_qp_offset+CuQpOffset Cr +actQpOffsetCr)+QpBdOffset

[0958] Qp' CbCr =Clip3(-QpBdOffset,63,

[0959] Q CbCr +pps_joint_cbcr_qp_offset+slice_joint_cbcr_qp_offset+CuQpOffset CbCr +actQpOffsetCbCr)+QpB dOffset

[0960] Next, a dequantization process of the transform coefficients may be performed, and the following information may be used as input to this process.

[0961] - Luma coordinates (xTbY, yTbY), which refer to the relative coordinates of the upper left sample of the current luma transform block relative to the upper left luma sample of the current picture,

[0962] - a variable nTbW indicating the width of the transform block,

[0963] - a variable nTbH indicating the height of the transform block,

[0964] - a variable predMode which indicates the prediction mode of the encoding basis,

[0965] - The variable cIdx, which indicates the color component of the current block

[0966] The output of the dequantization process of the transform coefficients may be an array d of scaled transform coefficients. Herein, the size of the array d may be (nTbW)×(nTbH). The individual elements constituting it may be identified as d[x][y].

[0967] When the present process is performed, the quantization parameter qP can be derived as follows: When cIdx has a value of 0, qP can be derived as shown in the following equation.

[0968] [Formula 119]

[0969] qP=Qp' Y

[0970] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, it can be derived as shown in the following equation.

[0971] [Formula 120]

[0972] qP=Qp' CbCr

[0973] Alternatively, when cIdx has a value of 1, qP may be derived as shown below.

[0974] [Formula 121]

[0975] qP=Qp' Cb

[0976] Alternatively, when cIdx has a value of 2, qP can be derived as shown in the following equation.

[0977] [Formula 122]

[0978] qP=Qp' Cr

[0979] The quantization parameter qP may be updated as follows. In addition, the variables rectNonTsFlag and bdShift may be derived as follows. When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, it may be derived as shown in the following equation.

[0980] [Formula 123]

[0981] rectNonTsFlag=(((Log2(nTbW)+Log2(nTbH))&1)==1)? 1:0

[0982] bdShift=BitDepth+rectNonTsFlag+((Log2(nTbW)+Log2(nTbH)) / 2)-5+pic_dep_quant_enabled_flag

[0983] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1 (eg, transform of the current transform block is skipped), it may be derived as shown in the following equation.

[0984] [Formula 124]

[0985] qP=Max(QpPrimeTsMin,qP)

[0986] rectNonTsFlag=0

[0987] bdShift=10

[0988] Encoding and decoding methods

[0989] In the following, reference will be made to Fig.52 and Fig.53 An image encoding method performed by an image encoding device and an image decoding method performed by an image decoding device are described.

[0990] First, the operation of the decoding device will be described. The image decoding device according to the embodiment may include a memory and a processor. The decoding device may perform decoding according to the operation of the processor. For example, Fig.52 As shown, in step S5210, the decoding device may determine the prediction mode of the current block based on the prediction information obtained from the bitstream. Next, in step S5220, the decoding device may obtain a flag indicating whether to apply color space transformation to the residual samples of the current block based on the prediction mode of the current block. Next, in step S5230, the decoding device may determine the quantization parameter of the current block based on the flag. Next, in step S5240, the decoding device may determine the transform coefficient of the current block based on the quantization parameter.

[0991] Herein, the decoding device may determine the quantization parameter by applying pruning to the quantization parameter so that the value of the quantization parameter has a value within a predetermined range. Herein, the lower limit value of the predetermined range may be 0. In addition, the upper limit value of the predetermined range may be determined based on a syntax element indicating the bit depth of the sample. For example, the upper limit value of the predetermined range may be determined as 63+QpBdOffset. QpBdOffset represents a quantization parameter range offset for luma and chroma, and may be preset to a predetermined constant or obtained from a bitstream. For example, QpBdOffset may be calculated by multiplying a predetermined constant by the value of a syntax element indicating the bit depth of a luma or chroma sample.

[0992] In addition, the decoding device may determine the quantization parameter based on the color component of the current block, may determine the quantization parameter offset based on the color component of the current block, and then may reset the quantization parameter by using the quantization parameter offset to determine the quantization parameter. For example, the decoding device may add the quantization parameter offset to the quantization parameter to reset the quantization parameter.

[0993] The quantization parameter offset may be determined as follows. When a color space transform is applied to the residual samples of the current block and the color component of the current block is a luminance component, the value of the quantization parameter offset may be determined to be -5. Alternatively, when a color space transform is applied to the residual samples of the current block and the color component of the current block is a chrominance Cb component, the value of the quantization parameter offset may be determined to be 1. Alternatively, when a color space transform is applied to the residual samples of the current block and the color component of the current block is a chrominance Cr component, the value of the quantization parameter offset may be determined to be 3.

[0994] In addition, the residual sample of the current block is determined based on the transform coefficient of the current block, and the value of the residual sample can be updated based on whether the color space transform is applied. For example, when the color space transform is applied, the value of the residual sample can be updated based on the sample value obtained by applying a right shift operation to the chroma component residual sample value. In an embodiment, the luminance component residual sample value of the residual sample can be updated by adding the value obtained by applying a right shift operation to the chroma Cb component residual sample value to the luminance component residual sample value. In addition, the chroma Cb component residual sample value of the residual sample can be updated with a value obtained by subtracting a first value and a second value from the luminance component residual sample value, wherein the first value is obtained by applying a right shift operation to the chroma Cb component residual sample value, and the second value is obtained by applying a right shift operation to the chroma Cr component residual sample value.

[0995] Hereinafter, the operation of the encoding device will be described. The image encoding device according to the embodiment may include a memory and a processor. The encoding device may perform encoding according to the operation of the processor in a manner corresponding to the decoding of the decoding device. For example, Fig.53As shown, in step S5310, the encoding device may determine the prediction mode of the current block. Next, in step S5320, the encoding device may determine the quantization parameter of the current block according to whether the color space transform is applied to the residual samples of the current block. Next, in step S5330, the encoding device may determine the transform coefficient of the current block based on the quantization parameter. Next, in step S5340, the encoding device may encode information indicating whether the color space transform is applied to the residual samples of the current block based on the prediction mode of the current block.

[0996] Corresponding to the operation of the above-mentioned decoding device, the encoding device can determine the quantization parameter by applying pruning to the quantization parameter so that the value of the quantization parameter has a value within a predetermined range. Herein, the lower limit value of the predetermined range may be 0. In addition, the upper limit value of the predetermined range may be set to a predetermined upper limit value. The upper limit value may be determined based on the bit depth of the sample. For example, the upper limit value of the predetermined range may be determined as 63+QpBdOffset. Herein, QpBdOffset represents the luminance and chrominance quantization parameter range offset, and may be preset to a predetermined constant or obtained from a bitstream. For example, QpBdOffset may be calculated by multiplying a predetermined constant by the value of a syntax element indicating the bit depth of a luminance or chrominance sample.

[0997] In addition, the encoding device may determine the quantization parameter based on the color component of the current block, may determine the quantization parameter offset based on the color component of the current block, and then may reset the quantization parameter by using the quantization parameter offset to determine the quantization parameter. For example, the encoding device may add the quantization parameter offset to the quantization parameter to reset the quantization parameter.

[0998] The quantization parameter offset may be determined as follows. When a color space transform is applied to the residual samples of the current block and the color component of the current block is a luminance component, the value of the quantization parameter offset may be determined to be -5. Alternatively, when a color space transform is applied to the residual samples of the current block and the color component of the current block is a chrominance Cb component, the value of the quantization parameter offset may be determined to be 1. Alternatively, when a color space transform is applied to the residual samples of the current block and the color component of the current block is a chrominance Cr component, the value of the quantization parameter offset may be determined to be 3.

[0999] In addition, the residual samples of the current block are determined based on the transform coefficients of the current block, and the values ​​of the residual samples can be updated based on whether the color space transform is applied. For example, when the color space transform is applied, the values ​​of the residual samples can be updated based on the sample values ​​obtained by applying a right shift operation to the chrominance component residual sample values.

[1000] Application Implementation

[1001] Although the exemplary method of the present disclosure described above is represented as a series of operations for the sake of clarity of description, it is not intended to limit the order of executing the steps, and these steps can be performed simultaneously or in different orders when necessary. In order to implement the method according to the present disclosure, the steps described may also include other steps, may include the remaining steps except some steps, or may include other additional steps except some steps.

[1002] In the present disclosure, an image encoding device or an image decoding device that performs a predetermined operation (step) may perform an operation (step) of confirming an execution condition or situation of the corresponding operation (step). For example, if it is described that a predetermined operation is performed when a predetermined condition is met, the image encoding device or the image decoding device may perform the predetermined operation after determining whether the predetermined condition is met.

[1003] The various embodiments of the present disclosure are not a list of all possible combinations and are intended to describe representative aspects of the present disclosure, and matters described in the various embodiments may be applied independently or in combination of two or more.

[1004] Various embodiments of the present disclosure may be implemented in hardware, firmware, software or a combination thereof. In the case of implementing the present disclosure in hardware, the present disclosure may be implemented using an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a general purpose processor, a controller, a microcontroller, a microprocessor, etc.

[1005] In addition, the image decoding device and the image encoding device to which the embodiments of the present disclosure are applied may be included in a multimedia broadcast transmission and reception device, a mobile communication terminal, a home theater video device, a digital theater video device, a surveillance camera, a video chat device, a real-time communication device such as video communication, a mobile streaming device, a storage medium, a camera, a video on demand (VoD) service provider, an OTT video (over the top video) device, an Internet streaming service provider, a three-dimensional (3D) video device, a video phone video device, a medical video device, etc., and may be used to process a video signal or a data signal. For example, an OTT video device may include a game console, a Blu-ray player, an Internet access TV, a home theater system, a smart phone, a tablet PC, a digital video recorder (DVR), etc.

[1006] Fig.54 is a diagram showing a content streaming system to which an embodiment of the present disclosure can be applied.

[1007] like Fig.54As shown in , a content streaming system to which the embodiments of the present disclosure are applied may mainly include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[1008] The encoding server compresses the content input from the multimedia input device such as a smart phone, camera, camcorder, etc. into digital data to generate a bit stream and sends the bit stream to the streaming server. As another example, when the multimedia input device such as a smart phone, camera, camcorder, etc. directly generates a bit stream, the encoding server can be omitted.

[1009] A bitstream may be generated by an image encoding method or an image encoding device to which an embodiment of the present disclosure is applied, and a streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[1010] The streaming server sends multimedia data to the user device based on the user's request through the network server, and the network server is used as a medium to inform the user of the service. When the user requests the required service from the network server, the network server can deliver it to the streaming server, and the streaming server can send the multimedia data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server is used to control the command / response between the devices in the content streaming system.

[1011] The streaming server may receive content from a media storage device and / or an encoding server. For example, when receiving content from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bitstream within a predetermined time.

[1012] Examples of user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays), digital televisions, desktop computers, digital signage, etc.

[1013] Each server in the content streaming system may operate as a distributed server, in which case data received from each server may be distributed.

[1014] The scope of the present disclosure includes software or machine-executable commands (e.g., operating systems, applications, firmware, programs, etc.) for enabling operations of methods according to various embodiments to be performed on a device or computer, and non-transitory computer-readable media having such software or commands stored thereon and executable on a device or computer.

[1015] Industrial Applicability

[1016] The embodiments of the present disclosure may be used to encode or decode an image.

Claims

1. A method for decoding an image, the method comprising the following steps: obtaining color space transform information for applying color space transform to residual samples of a current coding unit based on that the prediction mode is an intra prediction mode and the tree type is a single tree type; deriving a first quantization parameter of the current coding unit; deriving a final quantization parameter based on adding a value of the first quantization parameter of the current coding unit to a value of a quantization parameter offset of the current coding unit; deriving a transform coefficient of the current coding unit based on the final quantization parameter of the current coding unit; as well as deriving the residual samples of the current coding unit based on the transform coefficients of the current coding unit; The current coding unit includes a brightness component block, a first color component block, and a second color component block. Wherein, based on the color space transformation information indicating that the color space transformation is not applied to the residual sample of the current coding unit, the values ​​of the quantization parameter offsets of the luminance component block, the first color component block, and the second color component block of the current coding unit are respectively derived as 0, Wherein, based on the color space transformation information indicating application of the color space transformation to the residual samples of the current coding unit: The values ​​of the quantization parameter offsets of the luminance component block, the first color component block, and the second color component block of the current coding unit are derived to be different from each other, updating the values ​​of the residual samples of the luminance component block, the first color component block, and the second color component block respectively based on the values ​​of the residual samples of the luminance component block, updating the value of the residual sample of the first color component block based on subtracting a first value and a second value from the value of the residual sample of the luma component block, wherein the first value is a value obtained by applying a right shift operation to the value of the residual sample of the first color component block, and the second value is a value obtained by applying a right shift operation to the value of the residual sample of the second color component block, The values ​​of the residual samples of the second color component block are updated based on adding the updated values ​​of the residual samples of the first color component block to the values ​​of the residual samples of the second color component block.

2. The method according to claim 1, in, Based on color space transform information indicating that the color space transform is applied to the residual samples of the current coding unit, a value of the quantization parameter offset of the luma component block is derived as a negative value.

3. The method according to claim 1, in, Based on color space transform information indicating that the color space transform is applied to the residual samples of the current coding unit, the value of the quantization parameter offset of the luma component block is derived as -5.

4. The method according to claim 1, in, The value of the quantization parameter offset of the first color component block is derived to be 1 based on color space transform information indicating that the color space transform is applied to the residual samples of the current coding unit.

5. The method according to claim 1, in, Based on color space transform information indicating that the color space transform is applied to the residual samples of the current coding unit, the value of the quantization parameter offset of the second color component block is derived as 3.

6. The method according to claim 1, in, The first color component block is a chrominance Cb block, and the second color component block is a chrominance Cr block.

7. The method according to claim 1, in, Based on color space transformation information indicating that the color space transformation is applied to the residual samples of the current coding unit, the values ​​of the residual samples of the luma component block, the first color component block, and the second color component block are further updated based on sample values ​​obtained by applying the right shift operation to the values ​​of the residual samples of the first color component block.

8. The method according to claim 7, in, The sample value obtained by applying the right shift operation to the value of the residual sample of the first color component block corresponds to a half value of the residual sample of the first color component block.

9. The method according to claim 1, in, Based on color space transformation information indicating that the color space transformation is applied to the residual samples of the current coding unit, the values ​​of the residual samples of the luminance component block are updated based on adding sample values ​​obtained by applying the right shift operation to the values ​​of the residual samples of the first color component block and the values ​​of the residual samples of the luminance component block.

10. The method according to claim 9, in, A sample value obtained by applying the right shift operation to a value of the residual sample of the first color component block corresponds to a half value of the residual sample of the first color component block.

11. A method for encoding an image, the method comprising the following steps: deriving a color space transform applied to residual samples of a current coding unit based on the prediction mode being an intra prediction mode and the tree type being a single tree type; deriving residual samples for the current coding unit based on the applied color space transform; deriving transform coefficients of the residual samples of the current coding unit; as well as performing quantization based on a final quantization parameter of the current coding unit and the transform coefficient, wherein color space transform information for applying a color space transform to the residual samples of the current coding unit is encoded in a bitstream, The final quantization parameter is derived based on adding the value of the first quantization parameter of the current coding unit to the value of the quantization parameter offset of the current coding unit, The current coding unit includes a brightness component block, a first color component block, and a second color component block. Wherein, based on the color space transformation information indicating that the color space transformation is not applied to the residual sample of the current coding unit, the values ​​of the quantization parameter offsets of the luminance component block, the first color component block, and the second color component block of the current coding unit are respectively derived as 0, Wherein, based on the color space transformation information indicating application of the color space transformation to the residual samples of the current coding unit: The values ​​of the quantization parameter offsets of the luminance component block, the first color component block, and the second color component block of the current coding unit are derived to be different from each other, updating the values ​​of the residual samples of the luminance component block, the first color component block, and the second color component block respectively based on the values ​​of the residual samples of the luminance component block, updating the value of the residual sample of the first color component block based on subtracting a first value and a second value from the value of the residual sample of the luma component block, wherein the first value is a value obtained by applying a right shift operation to the value of the residual sample of the first color component block, and the second value is a value obtained by applying a right shift operation to the value of the residual sample of the second color component block, The values ​​of the residual samples of the second color component block are updated based on adding the updated values ​​of the residual samples of the first color component block to the values ​​of the residual samples of the second color component block.

12. A method for transmitting a bit stream, the method comprising the steps of: deriving a color space transform applied to residual samples of a current coding unit based on the prediction mode being an intra prediction mode and the tree type being a single tree type; deriving residual samples for the current coding unit based on the applied color space transform; deriving transform coefficients of the residual samples of the current coding unit; as well as performing quantization based on a final quantization parameter of the current coding unit and the transform coefficient, wherein color space transform information for applying a color space transform to the residual samples of the current coding unit is encoded in a bitstream, The final quantization parameter is derived based on adding the value of the first quantization parameter of the current coding unit to the value of the quantization parameter offset of the current coding unit, The current coding unit includes a brightness component block, a first color component block, and a second color component block. Wherein, based on the color space transformation information indicating that the color space transformation is not applied to the residual sample of the current coding unit, the values ​​of the quantization parameter offsets of the luminance component block, the first color component block, and the second color component block of the current coding unit are respectively derived as 0, Wherein, based on the color space transformation information indicating application of the color space transformation to the residual samples of the current coding unit: The values ​​of the quantization parameter offsets of the luminance component block, the first color component block, and the second color component block of the current coding unit are derived to be different from each other, updating the values ​​of the residual samples of the luminance component block, the first color component block, and the second color component block respectively based on the values ​​of the residual samples of the luminance component block, updating the value of the residual sample of the first color component block based on subtracting a first value and a second value from the value of the residual sample of the luma component block, wherein the first value is a value obtained by applying a right shift operation to the value of the residual sample of the first color component block, and the second value is a value obtained by applying a right shift operation to the value of the residual sample of the second color component block, The values ​​of the residual samples of the second color component block are updated based on adding the updated values ​​of the residual samples of the first color component block to the values ​​of the residual samples of the second color component block.

Images