Image decoding method, image encoding method, and bit stream transmission method

Abstract

The application relates to an image decoding method, an image encoding method, and a bitstream transmission method. An image encoding / decoding method and apparatus are provided. An image decoding method performed by an image decoding device according to the present disclosure comprises the steps of: determining a quantization parameter of a current block on the basis of whether color space conversion is applied to a residual sample of the current block; determining a conversion coefficient of the current block based on the quantization parameter; determining a residual sample of the current block using the conversion coefficient; and resetting the value of the residual sample based on whether the color-space conversion is applied.

Description

[0001] This application is a divisional application of the original invention patent application No. 202080090629.3 (International Application No.: PCT / KR2020 / 014844, Application Date: October 28, 2020, Invention Title: Method and Apparatus for Encoding / Decoding Images Using Color Space Conversion and Method for Transmitting Bit Streams). Technical Field

[0002] This disclosure relates to image encoding / decoding methods and apparatus. More specifically, this disclosure relates to an image encoding / decoding method and apparatus using color space conversion (or transformation), and a method for transmitting a bitstream generated by the image encoding method / apparatus of this disclosure. Background Technology

[0003] Recently, there has been an increasing demand for high-resolution and high-quality images, such as high-definition (HD) and ultra-high-definition (UHD) images, across various fields. With the improvement in image data resolution and quality, the amount of information or bits transmitted increases relatively compared to existing image data. This increase in the amount of information or bits transmitted leads to increased transmission and storage costs.

[0004] Therefore, efficient image compression techniques are needed to effectively transmit, store, and reproduce information about high-resolution and high-quality images. Summary of the Invention

[0005] Technical issues

[0006] This disclosure aims to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0007] In addition, this disclosure aims to provide an image encoding / decoding method and apparatus that improves encoding / decoding efficiency by performing selective color space conversion.

[0008] In addition, this disclosure aims to provide a method for transmitting a bit stream generated by an image encoding method or apparatus according to this disclosure.

[0009] In addition, this disclosure aims to provide a recording medium storing a bitstream generated by an image encoding method or apparatus according to this disclosure.

[0010] In addition, this disclosure aims to provide a recording medium storing a bitstream received, decoded and used to reconstruct an image by an image decoding apparatus according to this disclosure.

[0011] Those skilled in the art will understand that the technical objectives to be achieved by this disclosure are not limited to the above-described technical objectives, and other technical objectives not described herein will be clearly understood from the following description.

[0012] Technical solution

[0013] According to one aspect of this disclosure, an image decoding method performed by an image decoding apparatus is provided, the method comprising the steps of: determining a quantization parameter of a current block based on whether a color space transformation is applied to a residual sample of the current block; determining transform coefficients of the current block based on the quantization parameter; determining a residual sample of the current block using the transform coefficients; and resetting the value of the residual sample based on whether a color space transformation is applied. In this document, the step of determining the quantization parameter can be performed by cropping the quantization parameter such that the value of the quantization parameter has a value less than or equal to a predetermined upper limit and greater than or equal to a predetermined lower limit.

[0014] Furthermore, according to another aspect of this disclosure, an image decoding apparatus including a memory and at least one processor is provided. The at least one processor is configured to determine quantization parameters of a current block based on whether a color space transformation is applied to residual samples of the current block; determine transform coefficients of the current block based on the quantization parameters; determine residual samples of the current block using the transform coefficients; and reset the values ​​of the residual samples based on whether a color space transformation is applied. In this document, the processor may be configured to perform clipping on the quantization parameters such that the values ​​of the quantization parameters have values ​​less than or equal to a predetermined upper limit and greater than or equal to a predetermined lower limit.

[0015] Furthermore, according to another aspect of this disclosure, an image encoding method performed by an image encoding apparatus is provided, the method comprising the steps of: resetting residual samples based on whether a color space conversion is applied; determining transform coefficients using the reset residual samples; determining quantization parameters based on whether a color space conversion is applied; and encoding the transform coefficients based on the quantization parameters. In this document, the step of determining the quantization parameters can be performed by cropping the quantization parameters such that the values ​​of the quantization parameters have values ​​less than or equal to a predetermined upper limit and greater than or equal to a predetermined lower limit.

[0016] In addition, according to another aspect of this disclosure, a transmission method for transmitting a bit stream generated by the image encoding apparatus or image encoding method of this disclosure is provided.

[0017] In addition, according to another aspect of this disclosure, a computer-readable recording medium is provided that stores a bitstream generated by the image encoding method or image encoding apparatus of this disclosure.

[0018] The features described above in this brief overview are merely exemplary aspects of the following detailed description of this disclosure and do not limit the scope of this disclosure.

[0019] Beneficial effects

[0020] According to this disclosure, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency can be provided.

[0021] Furthermore, according to this disclosure, an image encoding / decoding method and apparatus capable of improving encoding / decoding efficiency by performing selective color space conversion can be provided.

[0022] Furthermore, according to this disclosure, a method for transmitting a bitstream generated by an image encoding method or apparatus according to this disclosure can be provided.

[0023] Furthermore, according to this disclosure, it is possible to provide a recording medium storing a bitstream generated by an image encoding method or apparatus according to this disclosure.

[0024] Furthermore, according to this disclosure, it is possible to provide a recording medium storing a bitstream that is received, decoded, and used to reconstruct an image by an image decoding apparatus according to this disclosure.

[0025] Those skilled in the art will understand that the effects achievable through this disclosure are not limited to those specifically described above, and that other advantages of this disclosure will become clearer from the following description. Attached Figure Description

[0026] Figure 1 This is a view schematically illustrating a video encoding system to which embodiments of this disclosure are applicable.

[0027] Figure 2 This is a schematic view illustrating an image encoding apparatus to which embodiments of the present disclosure are applicable.

[0028] Figure 3 This is a schematic view illustrating an image decoding apparatus to which embodiments of the present disclosure are applicable.

[0029] Figure 4 This is a view showing the segmentation structure of an image according to an embodiment.

[0030] Figure 5 This is a view illustrating an implementation of the block partitioning type based on a multi-type tree structure.

[0031] Figure 6 This is a view illustrating the signaling mechanism for block partitioning information in a quadtree structure with nested multi-type trees according to this disclosure.

[0032] Figure 7 This is a view illustrating an implementation of dividing a CTU into multiple CUs.

[0033] Figure 8 This is a view showing a block diagram of CABAC according to an implementation method for encoding a syntax element.

[0034] Figures 9 to 12This is a view illustrating entropy encoding and decoding according to an implementation method.

[0035] Figure 13 and Figure 14 This is a view illustrating an example of the screen decoding and encoding process according to an implementation method.

[0036] Figure 15 This is a view showing the hierarchical structure of the encoded image according to the embodiment.

[0037] Figure 16 This is a view showing a neighboring reference sample according to an embodiment.

[0038] Figure 17 and Figure 18 This is a view illustrating intra-frame prediction according to an implementation method.

[0039] Figure 19 This is a view illustrating an implementation of the decoding process using ACT.

[0040] Figure 20 This is a view illustrating an implementation of a sequence parameter set syntax table that uses signals to notify grammatical elements related to ACT.

[0041] Figures 21 to 27 It is a view that continuously shows an implementation of a syntax table that uses signals to notify the grammatical elements related to ACT.

[0042] Figure 28 This is a view showing the encoding tree syntax according to the implementation method.

[0043] Figure 29 This is a view illustrating the encoding method of residual samples of BDPCM according to an embodiment.

[0044] Figure 30 This is a view showing a modified quantization residual block generated by performing BDPCM according to an embodiment.

[0045] Figure 31 This is a flowchart illustrating the process of encoding the current block by applying BDPCM to an image encoding apparatus according to an embodiment.

[0046] Figure 32 This is a flowchart illustrating the process of reconstructing the current block by applying BDPCM in an image decoding device according to an embodiment.

[0047] Figures 33 to 35 This is a schematic view illustrating the syntax of using signals to communicate information about BDPCM.

[0048] Figures 36 to 51This is a view illustrating a syntax table for signaling ACT syntax elements according to various embodiments of the present disclosure.

[0049] Figure 52 This is a view illustrating an image decoding method according to an embodiment.

[0050] Figure 53 This is a view illustrating an image encoding method according to an embodiment.

[0051] Figure 54 This is a view illustrating the content streaming system to which embodiments of this disclosure are applicable. Detailed Implementation

[0052] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings to facilitate implementation by those skilled in the art. However, this disclosure can be implemented in various different forms and is not limited to the embodiments described herein.

[0053] In describing this disclosure, detailed descriptions of relevant known functions or constructions will be omitted if they unnecessarily obscure the scope of this disclosure. In the accompanying drawings, portions irrelevant to the description of this disclosure are omitted, and similar reference numerals are assigned to similar portions.

[0054] In this disclosure, when a component is "connected," "coupled," or "linked" to another component, it may include not only direct connections but also indirect connections where intermediate components exist. Furthermore, when a component "comprises" or "has" other components, unless otherwise stated, it means that other components may be included, not excluded.

[0055] In this disclosure, the terms first, second, etc., are used only for the purpose of distinguishing one component from other components and do not limit the order or importance of the components, unless otherwise stated. Accordingly, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment may be referred to as a first component in another embodiment.

[0056] In this disclosure, the components are distinguished from each other to clearly describe each feature, but this does not mean that the components must be separate. That is, multiple components may be integrated into a single hardware or software unit, or a single component may be distributed and implemented across multiple hardware or software units. Therefore, unless otherwise specified, implementations of these integrated or distributed components are included within the scope of this disclosure.

[0057] In this disclosure, the components described in the various embodiments are not necessarily essential components, and some components may be optional. Therefore, embodiments consisting of a subset of the components described in the embodiments are also included within the scope of this disclosure. Furthermore, embodiments that include other components besides those described in the various embodiments are also included within the scope of this disclosure.

[0058] This disclosure relates to the encoding and decoding of images. Unless redefined in this disclosure, the terms used herein may have the general meaning commonly used in the art to which this disclosure pertains.

[0059] In this disclosure, "video" can mean a collection of images over time. "Frame" generally refers to the basis representing an image within a specific time period, and a slice / piece is the coding basis that constitutes part of a frame in the encoding. A frame can consist of one or more slices / pieces. Additionally, a slice / piece can include one or more coding tree units (CTUs). A frame can consist of one or more slices / pieces. A frame can consist of one or more groups of pieces. A group of pieces can include one or more pieces. A tile can refer to a quadrilateral region of a CTU row within a piece in a frame. A piece can include one or more tiles. A tile can refer to a quadrilateral region of a CTU row within a piece. A piece can be divided into multiple tiles, each tile including one or more CTU rows belonging to the piece. A piece that is not divided into multiple tiles can also be considered a tile.

[0060] In this disclosure, "pixel" or "pixel" can refer to the smallest unit that constitutes a frame (or image). Furthermore, "sample" can be used as a term corresponding to a pixel. A sample can generally represent a pixel or a pixel value, or it can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.

[0061] In this disclosure, "unit" can refer to a basic unit of image processing. A unit may include a specific region of an image and at least one of the information associated with that region. A unit may include a luminance block and two chrominance (e.g., Cb, Cr) blocks. In some cases, the term "unit" may be used interchangeably with terms such as "sample array," "block," or "region." Generally, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients in M ​​columns and N rows.

[0062] In this disclosure, "current block" can mean one of "current coding block," "current coding unit," "coding target block," "decoding target block," or "processing target block." When performing prediction, "current block" can mean "current prediction block" or "prediction target block." When performing transform (inverse transform) / quantization (dequantization), "current block" can mean "current transform block" or "transform target block." When performing filtering, "current block" can mean "filter target block."

[0063] Furthermore, in this disclosure, unless explicitly stated as a chroma block, "current block" may mean "the luminance block of the current block". "The chroma block of the current block" can be expressed by including an explicit description of a chroma block such as "chroma block" or "current chroma block".

[0064] In this disclosure, the forward slash " / " or "," can be interpreted as indicating "and / or". For example, "A / B" and "A, B" can mean "A and / or B". Furthermore, "A / B / C" and "A, B, C" can mean "at least one of A, B and / or C".

[0065] In this disclosure, the term "or" should be interpreted to indicate "and / or". For example, the expression "A or B" can include 1) only "A", 2) only "B", or 3) both "A and B". In other words, in this disclosure, "or" should be interpreted to indicate "additionally or alternatively".

[0066] Overview of Video Encoding Systems

[0067] Figure 1 This is a schematic view of a video encoding system according to the present disclosure.

[0068] The video encoding system according to the embodiment may include a source device 10 and a receiving device 20. The source device 10 may deliver encoded video and / or image information or data to the receiving device 20 in the form of a file or stream via a digital storage medium or network.

[0069] The source device 10 according to the embodiment may include a video source generator 11, an encoding device 12, and a transmitter 13. The receiving device 20 according to the embodiment may include a receiver 21, a decoding device 22, and a renderer 23. The encoding device 12 may be referred to as a video / image encoding device, and the decoding device 22 may be referred to as a video / image decoding device. The transmitter 13 may be included in the encoding device 12. The receiver 21 may be included in the decoding device 22. The renderer 23 may include a display, and the display may be configured as a separate device or an external component.

[0070] The video source generator 11 can acquire video / images through a process of capturing, compositing, or generating video / images. The video source generator 11 may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive including previously captured video / images, etc. The video / image generation device may include, for example, a computer, tablet computer, and smartphone, and can generate video / images (electronically). For example, virtual video / images can be generated by a computer, etc. In this case, the video / image capture process can be replaced by a process of generating related data.

[0071] Encoding device 12 can encode input video / images. For compression and encoding efficiency, encoding device 12 can perform a series of processes such as prediction, transformation, and quantization. Encoding device 12 can output encoded data (encoded video / image information) in the form of a bitstream.

[0072] Transmitter 13 can transmit encoded video / image information or data, output in bitstream form, to receiver 21 of receiving device 20 in the form of a file or stream via digital storage medium or network. Digital storage medium can include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. Transmitter 13 can include elements for generating media files according to a predetermined file format and may include elements for transmission via broadcast / communication network. Receiver 21 can extract / receive bitstreams from storage medium or network and transmit the bitstreams to decoding device 22.

[0073] Decoding device 22 can decode video / images by performing a series of processes corresponding to the operation of encoding device 12, such as dequantization, inverse transform, and prediction.

[0074] Renderer 23 can render decoded video / images. The rendered video / images can be displayed on a monitor.

[0075] Overview of Image Encoding Devices

[0076] Figure 2 This is a schematic view illustrating an image encoding device to which embodiments of this disclosure may be applied.

[0077] like Figure 2As shown, the image encoding device 100 may include an image segmenter 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-frame prediction unit 180, an intra-frame prediction unit 185, and an entropy encoder 190. The inter-frame prediction unit 180 and the intra-frame prediction unit 185 may be collectively referred to as "prediction units". The transformer 120, quantizer 130, dequantizer 140, and inverse transformer 150 may be included in a residual processor. The residual processor may also include a subtractor 115.

[0078] In some implementations, all or at least some of the components configuring the image encoding device 100 may be configured by a single hardware component (e.g., an encoder or a processor). Furthermore, the memory 170 may include a decoded screen buffer (DPB) and may be configured by a digital storage medium.

[0079] Image segmenter 110 can segment an input image (or picture or frame) input to image encoding device 100 into one or more processing units. For example, a processing unit may be called an encoding unit (CU). Encoding units can be obtained by recursively segmenting encoding tree units (CTUs) or maximum encoding units (LCUs) according to a quadtree / binary tree / tritree (QT / BT / TT) structure. For example, an encoding unit can be segmented into multiple encoding units of greater depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. For the segmentation of encoding units, a quadtree structure can be applied first, followed by a binary tree structure and / or a ternary tree structure. The encoding process according to this disclosure can be performed based on the final encoding unit that is no longer segmented. The maximum encoding unit can be used as the final encoding unit, or a deeper encoding unit obtained by segmenting the maximum encoding unit can be used as the final encoding unit. Here, the encoding process may include prediction, transformation, and reconstruction processes, which will be described later. As another example, the processing unit of the encoding process may be a prediction unit (PU) or a transformation unit (TU). Prediction units and transform units can be partitioned or segmented from the final coding unit. Prediction units can be sample prediction units, and transform units can be units used to derive transform coefficients and / or units used to derive residual signals from transform coefficients.

[0080] The prediction unit (inter-frame prediction unit 180 or intra-frame prediction unit 185) can perform prediction on the block to be processed (the current block) and generate a prediction block that includes prediction samples of the current block. The prediction unit can determine whether to apply intra-frame prediction or inter-frame prediction based on the current block or CU. The prediction unit can generate various information related to the prediction of the current block and transmit the generated information to the entropy encoder 190. The information about the prediction can be encoded in the entropy encoder 190 and output as a bitstream.

[0081] Intra-prediction unit 185 can predict the current block by referencing samples in the current frame. Depending on the intra-prediction mode and / or intra-prediction technique, the reference samples may be located among the neighbors of the current block or may be placed separately. Intra-prediction modes may include multiple non-directional modes and multiple directional modes. Non-directional modes may include, for example, DC mode and planar mode. Depending on the level of detail in the prediction direction, directional modes may include, for example, 33 or 65 directional prediction modes. However, this is merely an example, and more or fewer directional prediction modes may be used depending on the settings. Intra-prediction unit 185 can determine the prediction mode to be applied to the current block by using prediction modes applied to neighboring blocks.

[0082] The inter-frame prediction unit 180 can deduce the prediction block of the current block based on a reference block (reference sample array) specified by motion vectors on a reference frame. In this case, to reduce the amount of motion information transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current frame and temporally neighboring blocks existing in the reference frame. The reference frame including the reference block and the reference frame including the temporally neighboring block may be the same or different. The temporally neighboring block may be referred to as a juxtaposed reference block, a juxtaposed CU (colCU), etc. The reference frame including the temporally neighboring block may be referred to as a juxtaposed frame (colPic). For example, the inter-frame prediction unit 180 can configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate to use to deduce the motion vector and / or reference frame index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the inter-frame prediction unit 180 can use motion information from neighboring blocks as motion information for the current block. In skip mode, unlike merge mode, residual signals may not be transmitted. In motion vector prediction (MVP) mode, motion vectors from neighboring blocks can be used as motion vector predictors, and the motion vector of the current block can be signaled by encoding motion vector differences and indicators of the motion vector predictors. The motion vector difference can refer to the difference between the motion vector of the current block and the motion vector predictor.

[0083] The prediction unit can generate a prediction signal based on various prediction methods and techniques described below. For example, the prediction unit can apply not only intra-frame prediction or inter-frame prediction, but also both intra-frame prediction and inter-frame prediction simultaneously to predict the current block. A prediction method that simultaneously applies both intra-frame prediction and inter-frame prediction to predict the current block can be called Combined Inter-Frame and Intra-Frame Prediction (CIIP). Furthermore, the prediction unit can perform Intra-Frame Block Copy (IBC) to predict the current block. Intra-Frame Block Copy can be used for content image / video coding in games, for example, Screen Content Coding (SCC). IBC is a method of predicting the current frame using a previously reconstructed reference block in the current frame at a predetermined distance from the current block. When IBC is applied, the position of the reference block in the current frame can be encoded as a vector (block vector) corresponding to the predetermined distance. IBC essentially performs prediction in the current frame, but can be performed similarly to inter-frame prediction because the reference block is derived within the current frame. That is, IBC can use at least one of the inter-frame prediction techniques described in this disclosure.

[0084] The prediction signal generated by the prediction unit can be used to generate a reconstructed signal or a residual signal. Subtractor 115 can generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output from the prediction unit from the input image signal (original block or original sample array). The generated residual signal can be transmitted to converter 120.

[0085] Transformer 120 can generate transform coefficients by applying transform techniques to the residual signal. For example, the transform techniques may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is represented graphically. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. Furthermore, the transform processing can be applied to square pixel blocks of the same size or to blocks of variable size instead of square.

[0086] Quantizer 130 quantizes the transform coefficients and transmits them to entropy encoder 190. Entropy encoder 190 encodes the quantized signal (information about the quantized transform coefficients) and outputs a bitstream. The information about the quantized transform coefficients can be referred to as residual information. Quantizer 130 can rearrange the block-form quantized transform coefficients into a one-dimensional vector form based on the coefficient scan order, and generate information about the quantized transform coefficients based on the one-dimensional vector form of the quantized transform coefficients.

[0087] The entropy encoder 190 can perform various encoding methods, such as exponential Columbus coding, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoder 190 can encode, either together or separately, the information required for video / image reconstruction other than the quantization transform coefficients (e.g., values ​​of syntax elements). The encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream form at the Network Abstraction Layer (NAL) level. The video / image information may also include information about various parameter sets, such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Furthermore, the video / image information may also include general constraint information. The signaled information, transmitted information, and / or syntax elements described in this disclosure can be encoded and included in the bitstream through the above encoding process.

[0088] The bitstream can be transmitted over a network or stored in a digital storage medium. The network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmitter (not shown) for transmitting the signal output from the entropy encoder 190 and / or a storage unit (not shown) for storing the signal may be included as internal / external components of the image encoding device 100. Alternatively, a transmitter may be provided as a component of the entropy encoder 190.

[0089] The quantized transform coefficients output from quantizer 130 can be used to generate residual signals. For example, the residual signals (residual blocks or residual samples) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients through dequantizer 140 and inverse transformer 150.

[0090] Adder 155 adds the reconstructed residual signal to the prediction signal output from inter-frame prediction unit 180 or intra-frame prediction unit 185 to generate a reconstructed signal (reconstructed frame, reconstructed block, reconstructed sample array). If the block to be processed has no residual, such as in the case of applying skip mode, the prediction block can be used as a reconstructed block. Adder 155 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current frame, and can be used for inter-frame prediction of the next frame by filtering as described below.

[0091] Filter 160 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 160 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 170, specifically in the DPB of memory 170. Various filtering methods can include, for example, deblocking filtering, sample adaptive offsetting, adaptive loop filtering, bilateral filtering, etc. Filter 160 can generate various filtering-related information and transmit the generated information to entropy encoder 190, as described later in the description of each filtering method. The filtering-related information can be encoded by entropy encoder 190 and output as a bitstream.

[0092] The modified reconstructed frame transmitted to memory 170 can be used as a reference frame in inter-frame prediction unit 180. When inter-frame prediction is applied by image encoding device 100, prediction mismatch between image encoding device 100 and image decoding device can be avoided and coding efficiency can be improved.

[0093] The DPB of memory 170 can store modified reconstructed frames for use as reference frames in inter-frame prediction unit 180. Memory 170 can store motion information of blocks from which motion information in the current frame is derived (or encoded) and / or motion information of already reconstructed blocks in the frame. The stored motion information can be transmitted to inter-frame prediction unit 180 and used as motion information for spatially or temporally neighboring blocks. Memory 170 can store reconstructed samples of reconstructed blocks in the current frame and can transmit the reconstructed samples to intra-frame prediction unit 185.

[0094] Overview of image decoding devices

[0095] Figure 3 This is a schematic view illustrating an image decoding device to which embodiments of the present disclosure may be applied.

[0096] like Figure 3 As shown, the image decoding device 200 may include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame prediction unit 260, and an intra-frame prediction unit 265. The inter-frame prediction unit 260 and the intra-frame prediction unit 265 may be collectively referred to as "prediction units". The dequantizer 220 and the inverse transformer 230 may be included in a residual processor.

[0097] According to an implementation, all or at least some of the components of the image decoding device 200 can be configured by hardware components (e.g., a decoder or a processor). Furthermore, the memory 250 may include a decoded screen buffer (DPB) or may be configured by a digital storage medium.

[0098] The image decoding device 200, having received a bitstream including video / image information, can perform operations related to... Figure 2 The image is reconstructed by processing corresponding to the processing performed by the image encoding device 100. For example, the image decoding device 200 can perform decoding using a processing unit applied in the image encoding device. Therefore, the decoding processing unit can be, for example, an encoding unit. The encoding unit can be obtained by segmenting a coding tree unit or a maximum coding unit. The reconstructed image signal decoded and output by the image decoding device 200 can be reproduced by a reproduction device (not shown).

[0099] Image decoding device 200 can receive data in bitstream form from... Figure 2The signal output by the image encoding device. The received signal can be decoded by the entropy decoder 210. For example, the entropy decoder 210 can parse the bitstream to derive the information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). In addition, the video / image information may also include general constraint information. The image decoding device can also decode the picture based on the information about the parameter sets and / or general constraint information. The information and / or syntax elements notified / received by signals described in this disclosure can be decoded and obtained from the bitstream through a decoding process. For example, the entropy decoder 210 decodes the information in the bitstream based on encoding methods such as exponential Golomb coding, CAVLC, or CABAC, and outputs the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients of the residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element in the bitstream, determine the context model using information about the target syntax element, decoding information of neighboring blocks and the target block, or information about symbols / bins decoded in the previous stage, perform arithmetic decoding on the bins based on the determined context model by predicting the occurrence probability of the bins, and generate symbols corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method can update the context model after determining the context model by using the information of the decoded symbols / bins for the context model of the next symbol / bin. The prediction-related information in the information decoded by the entropy decoder 210 can be provided to the prediction units (inter-frame prediction unit 260 and intra-frame prediction unit 265), and the residual value of entropy decoding performed in the entropy decoder 210, i.e., the quantization transform coefficients and related parameter information, can be input to the dequantizer 220. In addition, the filtering information in the information decoded by the entropy decoder 210 can be provided to the filter 240. Furthermore, the receiver (not shown) for receiving signals output from the image encoding device may be further configured as an internal / external element of the image decoding device 200, or the receiver may be a component of the entropy decoder 210.

[0100] Furthermore, the image decoding apparatus according to this disclosure can be referred to as a video / image / screen decoding apparatus. The image decoding apparatus can be divided into an information decoder (video / image / screen information decoder) and a sample decoder (video / image / screen sample decoder). The information decoder may include an entropy decoder 210. The sample decoder may include at least one of a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame prediction unit 260, or an intra-frame prediction unit 265.

[0101] Dequantizer 220 can dequantize the quantized transform coefficients and output transform coefficients. Dequantizer 220 can rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order performed in the image encoding device. Dequantizer 220 can obtain transform coefficients by performing dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information).

[0102] The inverse transformer 230 can perform inverse transformation on the transformation coefficients to obtain the residual signal (residual block, residual sample array).

[0103] The prediction unit can perform prediction on the current block and generate a prediction block that includes prediction samples of the current block. The prediction unit can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the prediction information output from the entropy decoder 210, and can determine a specific intra-frame / inter-frame prediction mode (prediction technique).

[0104] Similar to that described in the prediction unit of the image coding device 100, the prediction unit can generate a prediction signal based on various prediction methods (techniques) described later.

[0105] Intra-prediction unit 265 can predict the current block by referring to samples in the current frame. The description of intra-prediction unit 185 also applies to intra-prediction unit 265.

[0106] The inter-frame prediction unit 260 can deduce the prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference frame. In this case, to reduce the amount of motion information transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current frame and temporally neighboring blocks existing in the reference frame. For example, the inter-frame prediction unit 260 can configure a motion information candidate list based on neighboring blocks and deduce the motion vector and / or reference frame index of the current block based on the received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the inter-frame prediction mode of the current block.

[0107] Adder 235 generates a reconstruction signal (reconstructed frame, reconstruction block, reconstruction sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the prediction unit (including inter-frame prediction unit 260 and / or intra-frame prediction unit 265). If the block to be processed has no residual, such as when a skip mode is applied, the prediction block can be used as a reconstruction block. The description of adder 155 also applies to adder 235. Adder 235 can be referred to as a reconstructor or reconstruction block generator. The generated reconstruction signal can be used for intra-frame prediction of the next block to be processed in the current frame, and can be used for inter-frame prediction of the next frame by filtering as described below.

[0108] Filter 240 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 240 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 250, specifically in the DPB of memory 250. Various filtering methods may include, for example, deblocking filtering, adaptive sample shifting, adaptive loop filtering, bilateral filtering, etc.

[0109] The (modified) reconstructed frame stored in the DPB of memory 250 can be used as a reference frame in inter-frame prediction unit 260. Memory 250 can store motion information of blocks from which motion information in the current frame is derived (or decoded) and / or motion information of already reconstructed blocks in the frame. The stored motion information can be transmitted to inter-frame prediction unit 260 to be used as motion information for spatially or temporally neighboring blocks. Memory 250 can store reconstructed samples of reconstructed blocks in the current frame and transmit the reconstructed samples to intra-frame prediction unit 265.

[0110] In this disclosure, the embodiments described in the filter 160, inter-frame prediction unit 180 and intra-frame prediction unit 185 of the image encoding device 100 can be equally or correspondingly applied to the filter 240, inter-frame prediction unit 260 and intra-frame prediction unit 265 of the image decoding device 200.

[0111] Overview of Image Segmentation

[0112] The video / image coding method according to this disclosure can be performed based on the image segmentation structure as follows. Specifically, the processes of prediction, residual processing (inverse transform, dequantization, etc.), syntax element encoding, and filtering, which will be described later, can be performed based on the CTU, CU (and / or TU, PU) derived from the image segmentation structure. The image can be segmented into block units, and the block segmentation process can be performed in the image segmenter 110 of the encoding device. The segmentation-related information can be encoded by the entropy encoder 190 and sent to the image decoding device in the form of a bitstream. The entropy decoder 210 of the image decoding device can deduce the block segmentation structure of the current frame based on the segmentation-related information obtained from the bitstream, and based on this, a series of processes (e.g., prediction, residual processing, block / frame reconstruction, in-loop filtering, etc.) can be performed for image decoding.

[0113] The image can be segmented into a sequence of coding tree units (CTUs). Figure 4 An example of a screen being segmented into CTUs is shown. A CTU may correspond to a Coding Tree Block (CTB). Alternatively, a CTU may include a coded tree block for luma samples and two coded tree blocks for corresponding chroma samples. For example, for a screen containing three sample arrays, a CTU may include an N×N block for luma samples and two corresponding blocks for chroma samples.

[0114] Overview of CTU segmentation

[0115] As described above, coding units can be obtained by recursively partitioning coding tree units (CTUs) or maximum coding units (LCUs) according to a quadtree / binary tree / tritree (QT / BT / TT) structure. For example, a CTU can be first partitioned into a quadtree structure. Subsequently, the leaf nodes of the quadtree structure can be further partitioned using multiple tree types.

[0116] The quadtree partitioning means that the current CU (or CTU) is equally divided into four. By partitioning according to the quadtree, the current CU can be divided into four CUs with the same width and height. When the current CU is no longer partitioned into a quadtree structure, the current CU corresponds to a leaf node of the quadtree structure. The CUs corresponding to the leaf nodes of the quadtree structure may not be further partitioned and can be used as the final encoding unit described above. Alternatively, the CUs corresponding to the leaf nodes of the quadtree structure can be further partitioned using multiple types of tree structures.

[0117] Figure 5 This is a view illustrating an implementation of block partitioning types based on multiple tree structures. Partitioning based on multiple tree structures can include two types of partitioning based on binary tree structures and two types of partitioning based on ternary tree structures.

[0118] The two types of partitioning based on the binary tree structure can be vertical binary partitioning (SPLIT_BT_VER) and horizontal binary partitioning (SPLIT_BT_HOR). Vertical binary partitioning (SPLIT_BT_VER) means that the current CU is equally divided into two in the vertical direction. For example... Figure 4 As shown, a vertical binary partition can generate two CUs with the same height as the current CU and a width half the width of the current CU. A horizontal binary partition (SPLIT_BT_HOR) means that the current CU is equally divided into two in the horizontal direction. Figure 5 As shown, by using horizontal binary partitioning, two CUs can be generated with a height that is half the height of the current CU and a width that is the same as the current CU.

[0119] The two types of partitioning based on the ternary number structure can include vertical ternary partitioning (SPLIT_TT_VER) and horizontal ternary partitioning (SPLIT_TT_HOR). In vertical ternary partitioning (SPLIT_TT_VER), the current CU is partitioned vertically in a 1:2:1 ratio. For example... Figure 5 As shown, a vertical truncation can generate two CUs with the same height as the current CU and a width one-quarter of the current CU's width, and one CU with the same height as the current CU and a width half of the current CU's width. In a horizontal truncation (SPLIT_TT_HOR), the current CU is divided horizontally in a 1:2:1 ratio. Figure 4 As shown, by dividing horizontally into three branches, two CUs with a height of 1 / 4 of the current CU's height and the same width as the current CU, and one CU with a height of half the current CU's height and the same width as the current CU, can be generated.

[0120] Figure 6 This is a view illustrating the signaling mechanism for block partitioning information in a quadtree structure with nested multi-type trees according to this disclosure.

[0121] Here, the CTU is considered the root node of the quadtree and is initially split into a quadtree structure. Information indicating whether to perform a quadtree split on the current CU (the CTU or node (QT_node) of the quadtree) is provided (e.g., qt_split_flag). For example, when qt_split_flag has a first value (e.g., "1"), the current CU can be split into the quadtree. Alternatively, when qt_split_flag has a second value (e.g., "0"), the current CU is not split into the quadtree but becomes a leaf node (QT_leaf_node). Each quadtree leaf node can then be further split into a multi-type tree structure. That is, a leaf node of the quadtree can become a node of a multi-type tree (MTT_node). In the multi-type tree structure, a first flag (e.g., Mtt_split_cu_flag) is used to indicate whether the current node is additionally split. If the corresponding node is additionally split (e.g., if the first flag is 1), a second flag (e.g., `Mtt_split_cu_vertical_flag`) can be signaled to indicate the split direction. For example, the split direction could be vertical when the second flag is 1, and horizontal when the second flag is 0. Then, a third flag (e.g., `Mtt_split_cu_binary_flag`) can be signaled to indicate whether the split type is binary or ternary. For example, the split type could be binary when the third flag is 1, and ternary when the third flag is 0. Nodes of the multi-type tree obtained through binary or ternary splits can be further split into multi-type tree structures. However, the nodes of the multi-type tree may not be split into quadtree structures. If the first flag is 0, the corresponding node of the multi-type tree is no longer split, but becomes a leaf node (`MTT_leaf_node`) of the multi-type tree. The CU corresponding to the leaf node of the multi-type tree can be used as the final encoding unit described above.

[0122] Based on `mtt_split_cu_vertical_flag` and `mtt_split_cu_binary_flag`, the multi-tree partitioning mode (MttSplitMode) of the CU can be derived as shown in Table 1 below. In the following description, the multi-tree partitioning mode can be simply referred to as the multi-tree partitioning type or partitioning type.

[0123] [Table 1]

[0124] Figure 7 This is a view illustrating an example of partitioning a CTU into multiple CUs by applying a multi-type tree after applying a quadtree. Figure 7In the diagram, bold border 710 represents quadtree segmentation, while remaining border 720 represents multi-type tree segmentation. A CU can correspond to a coded block (CB). In an implementation, a CU may include one coded block for luminance samples and two coded blocks for chrominance samples corresponding to the luminance samples. The size of the chrominance component (sample) CB or TB can be derived based on the component ratio of the color format (chrominance format, e.g., 4:4:4, 4:2:2, 4:2:0, etc.) of the image / picture, according to the luminance component (sample) CB or TB size. In the case of a 4:4:4 color format, the chrominance component CB / TB size can be set to be equal to the luminance component CB / TB size. In the case of a 4:2:2 color format, the width of the chrominance component CB / TB can be set to half the width of the luminance component CB / TB, and the height of the chrominance component CB / TB can be set to the height of the luminance component CB / TB. In the 4:2:0 color format, the width of the chroma component CB / TB can be set to half the width of the luminance component CB / TB, and the height of the chroma component CB / TB can be set to half the height of the luminance component CB / TB.

[0125] In one implementation, when the size of the CTU is based on 128 luminance sample units, the size of the CU can be from 128×128 to 4×4, which is the same size as the CTU. In one implementation, in the case of a 4:2:0 color format (or chroma format), the chroma CB size can be from 64×64 to 2×2.

[0126] Furthermore, in implementations, the CU size and TU size can be the same. Alternatively, there can be multiple TUs in the CU region. The TU size typically represents the size of the luminance component (sample) transform block (TB).

[0127] The TU size can be derived based on the maximum permissible TB size, maxTbSize, which is a predetermined value. For example, when the CU size is greater than maxTbSize, multiple TUs (TBs) with maxTbSize can be derived from the CU, and transformations / inverse transformations can be performed on TUs (TBs) as units. For example, the maximum permissible luminance TB size can be 64×64 and the maximum permissible chrominance TB size can be 32×32. If the width or height of the CB segmented according to the tree structure is greater than the maximum transformation width or height, the CB can be automatically (or implicitly) segmented until the TB size limits in the horizontal and vertical directions are met.

[0128] Additionally, for example, when applying intra-frame prediction, the intra-frame prediction mode / type can be derived at the CU (or CB) level, and the neighbor reference sample derivation and prediction sample generation process can be performed at the TU (or TB) level. In this case, there can be one or more TUs (or TBs) in a CU (or CB) region, and multiple TUs or (TBs) can share the same intra-frame prediction mode / type.

[0129] Furthermore, for quadtree coding schemes with nested multi-type trees, the following parameters can be signaled from the image encoding device to the decoding device as SPS syntax elements. For example, at least one of the following can be signaled: CTU size as a parameter representing the size of the root node of the quadtree; MinQTSize as a parameter representing the minimum allowed size of the leaf node of the quadtree; MaxBtSize as a parameter representing the maximum allowed size of the root node of the binary tree; MaxTtSize as a parameter representing the maximum allowed size of the root node of the ternary tree; MaxMttDepth as a parameter representing the maximum allowed depth of the multi-type tree partitioning starting from the leaf node of the quadtree; MinBtSize as a parameter representing the minimum allowed size of the leaf node of the binary tree; or MinTtSize as a parameter representing the minimum allowed size of the leaf node of the ternary tree.

[0130] As an implementation using the 4:2:0 chroma format, the CTU size can be set to 128×128 luma blocks and two corresponding 64×64 chroma blocks. In this case, MinOTSize can be set to 16×16, MaxBtSize to 128×128, MaxTtSzie to 64×64, MinBtSize and MinTtSize to 4×4, and MaxMttDepth to 4. Quadtree partitioning can be applied to the CTU to generate quadtree leaf nodes. Quadtree leaf nodes can be called leaf QT nodes. The size of quadtree leaf nodes can range from 16×16 (e.g., MinOTSize) to 128×128 (e.g., CTU size). If a leaf QT node is 128×128, it can be partitioned into a binary / ternary tree without additional partitioning. This is because, in this case, even if partitioned, it would exceed MaxBtsize and MaxTtszie (e.g., 64×64). In other cases, leaf QT nodes can be further segmented into multi-type trees. Therefore, a leaf QT node is the root node of the multi-type tree, and a leaf QT node can have a multi-type tree depth (mttDepth) of 0. If the multi-type tree depth reaches MaxMttdepth (e.g., 4), further segmentation can be disregarded. If the width of a multi-type tree node is equal to MinBtSize and less than or equal to 2 × MinTtSize, further horizontal segmentation can be disregarded. If the height of a multi-type tree node is equal to MinBtSize and less than or equal to 2 × MinTtSize, further vertical segmentation can be disregarded. When segmentation is disregarded, the image encoding device can skip the signaling of segmentation information. In this case, the image decoding device can deduce segmentation information with predetermined values.

[0131] Furthermore, a CTU can include a coded block of luma samples (hereinafter referred to as a "luma block") and two coded blocks of corresponding chroma samples (hereinafter referred to as "chroma blocks"). The above coding tree scheme can be applied equally or separately to the luma and chroma blocks of the current CU. Specifically, the luma and chroma blocks in a CTU can be partitioned into the same block tree structure, and in this case, the tree structure is represented as SINGLE_TREE. Alternatively, the luma and chroma blocks in a CTU can be partitioned into separate block tree structures, and in this case, the tree structure can be represented as DUAL_TREE. That is, when the CTU is partitioned into two trees, the block tree structure for the luma blocks and the block tree structure for the chroma blocks can exist separately. In this case, the block tree structure for the luma blocks can be called DUAL_TREE_LUMA, and the block tree structure for the chroma components can be called DUAL_TREE_CHROMA. For P and B slice / piece groups, the luma and chroma blocks in a CTU can be restricted to having the same coding tree structure. However, for I-slice / patch groups, luma blocks and chroma blocks can have separate block tree structures. If separate block tree structures are applied, the luma CTB can be segmented into CUs based on a specific coding tree structure, and the chroma CTB can be segmented into chroma CUs based on another coding tree structure. That is, this means that CUs in I-slice / patch groups with separate block tree structures can include either a coded block of the luma component or coded blocks of the two chroma components, and CUs in P or B-slice / patch groups can include blocks of three color components (one luma component and two chroma components).

[0132] Although quadtree coding tree structures with nested multi-type trees have been described, the structures for splitting CUs are not limited to this. For example, BT and TT structures can be interpreted as concepts included in a multi-split tree (MPT) structure, and CUs can be interpreted as being split via QT and MPT structures. In the example of splitting CUs via QT and MPT structures, the splitting structure can be determined by signaling syntax elements (e.g., MPT_split_type) that include information about how many blocks the leaf nodes of the QT structure are split into, and syntax elements (e.g., MPT_split_mode) that include information about which direction (vertical or horizontal) the leaf nodes of the QT structure are split into.

[0133] In another example, the CU can be segmented in a manner different from the QT, BT, or TT structures. That is, unlike the QT structure which segments a lower-depth CU into 1 / 4 of a higher-depth CU, the BT structure which segments a lower-depth CU into 1 / 2 of a higher-depth CU, or the TT structure which segments a lower-depth CU into 1 / 4 or 1 / 2 of a higher-depth CU, in some cases the lower-depth CU can be segmented into 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 of the higher-depth CU, and the method of segmenting the CU is not limited to these.

[0134] Quadtree coding block structures with multiple tree types can provide highly flexible block partitioning structures. Due to the partitioning types supported in multiple tree types, different partitioning patterns can potentially produce the same coding block structure in some cases. In image encoding and decoding devices, the amount of data for partitioning information can be reduced by limiting the occurrence of such redundant partitioning patterns.

[0135] Furthermore, in the encoding and decoding of video / images according to this document, the image processing foundation can have a hierarchical structure. A frame can be divided into one or more tiles, blocks, slices, and / or tile groups. A slice may include one or more blocks. A block may include one or more CTU rows within a tile. A slice may include blocks of the frame, wherein the number of blocks is an integer. A tile group may include one or more tiles. A tile may include one or more CTUs. A CTU may be divided into one or more CUs. A tile may be a quadrilateral region consisting of specific tile rows and specific tile columns including multiple CTUs within the frame. A tile group may include tiles raster scanned according to the tiles within the frame, wherein the number of tiles is an integer. A slice header may carry information / parameters applicable to the corresponding slice (blocks within a slice). When the encoding or decoding device has a multi-core processor, the encoding / decoding process for tiles, slices, blocks, and / or tile groups can be executed in parallel.

[0136] In this disclosure, the names or concepts of slice or tile group are used interchangeably. That is, a tile group header can be referred to as a slice header. In this document, a slice can have one of the slice types, including intra-frame (I) slices, prediction (P) slices, and dual-prediction (B) slices. For blocks within an I slice, inter-frame prediction is not used for prediction; only intra-frame prediction can be used. Even in this case, the original sample values ​​can be encoded and signaled without prediction. For blocks within a P slice, either intra-frame prediction or inter-frame prediction can be used. When using inter-frame prediction, only single prediction can be used. Furthermore, for blocks within a B slice, either intra-frame prediction or inter-frame prediction can be used. When using inter-frame prediction, at most dual prediction can be used.

[0137] Based on the characteristics of the video image (e.g., resolution) or considering coding efficiency or parallel processing, the encoding device can determine the tile / tile group, patch, slice, and the maximum and minimum coding unit size. Additionally, information about this, or information used to derive this, can be included in the bitstream.

[0138] Decoding devices can obtain information indicating whether a tile / tile group, patch, slice, or CTU within a tile in the current frame has been divided into multiple coding units. Encoding and decoding devices only signal this information under specific conditions, thereby increasing encoding efficiency.

[0139] A slice header (slice header syntax) can include information / parameters common to a slice. An APS (APS syntax) or PPS (PPS syntax) can include information / parameters common to one or more frames. An SPS (SPS syntax) can include information / parameters common to one or more sequences. A VPS (VPS syntax) can include information / parameters common to multiple layers. A DPS (DPS syntax) can include information / parameters common to the entire video. A DPS can include information / parameters related to the combination of encoded video sequences (CVS).

[0140] Additionally, information regarding the segmentation and configuration of tiles / tile groups / tiles / slices can be constructed using high-level syntax during the encoding phase and sent to the decoding device as a bitstream.

[0141] Quantization / Dequantization

[0142] As described above, the quantizer of the encoding device can derive the quantized transform coefficients by applying quantization to the transform coefficients. Similarly, the dequantizer of the encoding device or the dequantizer of the decoding device can derive the transform coefficients by applying dequantization to the quantized transform coefficients.

[0143] In video / still image encoding and decoding, the quantization ratio can be changed, and the compression ratio can be adjusted using the changed quantization ratio. From an implementation perspective, considering complexity, a quantization parameter (QP) can be used instead of directly using the quantization ratio. For example, a quantization parameter with integer values ​​from 0 to 63 can be used, and each quantization parameter value can correspond to the actual quantization ratio. Additionally, the quantization parameter QP for the luma component (luma sample) can be... Y Quantization parameter QP of chromaticity components (chromaticity samples) C They can be set to be different from each other.

[0144] In quantization, the transform coefficients C can be input and divided by the quantization ratio Qstep, and based on this, the quantization transform coefficients C' can be obtained. In this case, considering computational complexity, the quantization ratio can be multiplied by a scale to become an integer, and a shift operation can be performed based on the value corresponding to the scale value. Based on the multiplication of the quantization ratio and the scale value, the quantization scale can be derived. That is, the quantization scale can be derived from QP. The quantization scale can be applied to the transform coefficients C, and based on this, the quantization transform coefficients C' can be derived.

[0145] Dequantization is the inverse process of quantization. The quantized transform coefficients C' can be multiplied by the quantization ratio Qstep, and based on this, the reconstructed transform coefficients C'' can be obtained. In this case, the level scale can be derived from the quantization parameters. The level scale can be applied to the quantized transform coefficients C', and based on this, the reconstructed transform coefficients C'' can be derived. Due to losses in the transform and / or quantization processes, the reconstructed transform coefficients C'' may differ slightly from the original transform coefficients C. Therefore, the encoding device can perform dequantization in the same manner as the decoding device.

[0146] Furthermore, adaptive frequency-weighted quantization (IFQ) can be applied, where the quantization intensity is adjusted according to the frequency. IFQ is a method that applies a frequency-varying quantization intensity. In IFQ, a predefined quantization scaling matrix can be used to apply the frequency-varying quantization intensity. That is, the aforementioned quantization / dequantization processing can be further performed based on the quantization scaling matrix. For example, different quantization scaling matrices can be used depending on the size of the current block and / or whether the prediction mode applied to the current block to generate the residual signal of the current block is inter-frame prediction or intra-frame prediction. The quantization scaling matrix can be referred to as the quantization matrix or the scaling matrix. The quantization scaling matrix can be predefined. Additionally, for frequency-adaptive scaling, the frequency-specific quantization scaling information of the quantization scaling matrix can be constructed / encoded by the encoding device and signaled to the decoding device. This frequency-specific quantization scaling information can be referred to as quantization scaling information. The frequency-specific quantization scaling information can include scaling list data. The (modified) quantization scaling matrix can be derived based on the scaling list data. Furthermore, the frequency-specific quantization scaling information can include presence flags indicating the presence of scaling list data. Alternatively, when the scaling list data is signaled at a higher level (e.g., SPS), information indicating whether the scaling list data should be modified at a lower level (e.g., PPS or tile group head, etc.) may also be included.

[0147] Transform / Inverse Transform

[0148] As described above, the encoding device can derive residual blocks (residual samples) based on blocks (predicted samples) predicted via intra / inter / IBC prediction, and can derive quantization transform coefficients by applying transform and quantization to the derived residual samples. Information about the quantization transform coefficients (residual information) can be encoded and output as a bitstream by including it in the residual coding syntax. The decoding device can obtain information about the quantization transform coefficients (residual information) from the bitstream and can derive the quantization transform coefficients by performing decoding. The decoding device can derive residual samples based on the quantization transform coefficients by dequantizing / inverse transforming. As described above, quantization / dequantization or transform / inverse transform, or both, can be omitted. When transform / inverse transform is omitted, transform coefficients can be referred to as coefficients or residual coefficients, or, for consistency, can still be referred to as transform coefficients. Whether transform / inverse transform is omitted can be signaled based on a transform skip flag (e.g., transform_skip_flag). The first value of transform_skip_flag (e.g., 0) can indicate whether transform is omitted, determined by another syntax element. The second value of transform_skip_flag (e.g., 1) can indicate that the transformation is omitted (e.g., skipped).

[0149] Transform / inverse transforms can be performed based on transform kernels. For example, a multiple transform selection (MTS) scheme for performing transform / inverse transforms can be applied. In this case, some from a set of multiple transform kernels can be selected and applied to the current block. Transform kernels can be referred to by various terms such as transform matrix, transform type, etc. For example, a set of transform kernels can refer to a combination of vertical transform kernels (vertical transform kernels) and horizontal transform kernels (horizontal transform kernels).

[0150] Transform / inverse transform can be performed per CU or TU. That is, transform / inverse transform can be applied to residual samples in a CU or residual samples in a TU. The CU size and TU size can be the same, or multiple TUs can exist in a CU region. Furthermore, the CU size can typically refer to the size of the luma component (sample) CB. The TU size can typically refer to the size of the luma component (sample) TB. The chroma component (sample) CB or TB size can be derived based on the luma component (sample) CB or TB size according to the color format (chroma format, such as 4:4:4, 4:2:2, 4:2:0, etc.) according to the component ratio. The TU size can be derived based on maxTbSize. For example, when the CU size is greater than maxTbSize, multiple TUs (TBs) with maxTbSize can be derived from the CU, and transform / inverse transform can be performed per TU (TB). maxTbSize can be considered when determining whether to apply various intra-frame prediction types such as ISP. Information about maxTbSize can be reserved. Alternatively, information about maxTbSize can be generated and encoded by the encoding device and signaled to the encoding device.

[0151] Entropy coding

[0152] As shown above (refer to the reference) Figure 2 As described, some or all of the video / image information can be encoded by an entropy encoder with 190 entropy. (See reference...) Figure 3 Some or all of the video / image information described can be entropy decoded by the entropy decoder 210. In this case, the video / image information can be encoded / decoded based on each syntax element. In this document, the encoding / decoding of information can include encoding / decoding performed by the methods described in this paragraph.

[0153] Figure 8 This diagram illustrates a block diagram of CABAC used to encode a syntax element. In CABAC encoding, firstly, when the input signal is a non-binary syntax element, it is transformed into a binary value through binarization. When the input signal is already a binary value, binarization is bypassed. In this paper, the individual binary numbers 0 or 1 that constitute the binary value can be called bins. For example, when the binarized binary string (bin string) is 110, each of the 1s, 1s, and 0s is called a bin. The bin of a syntax element can refer to the value of the syntax element.

[0154] Binarized bins can be input into either a regular encoding engine or a bypass encoding engine. A regular encoding engine assigns a context model with applied probability values ​​to the corresponding bin and encodes the bin based on the assigned context model. After encoding each bin, the regular encoding engine updates the bin's probability model. Bins encoded in this way are called context-coded bins. A bypass encoding engine can omit the processes for estimating the probabilities of the input bins and updating the probability models applied to the bins after encoding. In the case of a bypass encoding engine, instead of assigning context, the input bins are encoded by applying a uniform probability distribution (e.g., 50:50), thereby improving the encoding rate. Bins encoded in this way are called bypass bins. Context models can be assigned and updated for each bin to be context-coded (regular encoding), and the context model can be indicated based on ctxidx or ctxInc. ctxidx can be derived based on ctxInc. Specifically, for example, the context index (ctxidx) indicating the context model for each regular encoded bin can be derived as the sum of the context index increment (ctxInc) and the context index offset (ctxIdxOffset). In this paper, ctxInc, which varies from bin to bin, can be derived. ctxIdxOffset can be represented by the lowest value of ctxIdx. The lowest value of ctxIdx can be called the initial value (initValue) of ctxIdx. ctxIdxOffset is a value typically used to distinguish the context model from other syntax elements, and the context model of a syntax element can be distinguished / derived based on ctxinc.

[0155] During entropy coding, it is determined whether to perform encoding through a regular encoding engine or a bypass encoding engine, and the encoding path can be switched. Entropy decoding can perform the same processing as entropy coding in reverse order.

[0156] For example, it can be like Figure 9 and Figure 10 The above entropy encoding is performed in the process. (Refer to...) Figure 9 and Figure 10 An encoding device (entropy encoder) can perform entropy coding on image / video information. Image / video information may include segmentation-related information, prediction-related information (e.g., inter-frame / intra-frame prediction classification information, intra-frame prediction mode information, and inter-frame prediction mode information), residual information, and in-loop filtering-related information, or may include various related syntax elements. Entropy coding can be performed on a per-syntax-element basis. Figure 9 Steps S910 to S920 can be performed by the above Figure 2 The entropy encoder 190 of the encoding device is executed.

[0157] In step S910, the encoding device may perform binarization on the target syntax element. Herein, binarization may be based on various binarization methods such as truncated Ricean binarization and fixed-length binarization, and the binarization method used for the target syntax element may be predefined. The binarization process may be performed by the binarization unit 191 in the entropy encoder 190.

[0158] In step S920, the encoding device may perform entropy encoding on the target syntax element. The encoding device may perform regular encoding (context-based) or bypass encoding on the bin string of the target syntax element based on an entropy encoding scheme such as Context Adaptive Arithmetic Coding (CABAC) or Context Adaptive Variable Length Coding (CAVLC). The output may be included in the bitstream. The entropy encoding process may be executed by the entropy encoding processor 192 in the entropy encoder 190. As described above, the bitstream may be sent to the decoding device via a (digital) storage medium or network.

[0159] Reference Figure 11 and Figure 12 The decoding device (entropy decoder) can decode the encoded image / video information. The image / video information may include segmentation-related information, prediction-related information (e.g., inter-frame / intra-frame prediction classification information, intra-frame prediction mode information, and inter-frame prediction mode information), residual information, and in-loop filtering-related information, or may include various related syntax elements. Entropy coding can be performed on a per-syntax element basis. Steps S1110 to S1120 can be performed as described above. Figure 3 The entropy decoder 210 of the decoding device is executed.

[0160] In step S1110, the decoding device may perform binarization on the target syntax element. Herein, binarization may be based on various binarization methods such as truncated Ricean binarization and fixed-length binarization, and the binarization method used for the target syntax element may be predefined. The decoding device may derive available bin strings (bin string candidates) of available values ​​for the target syntax element through the binarization process. The binarization process may be performed by the binarization unit 211 in the entropy decoder 210.

[0161] In step S1120, the decoding device can perform entropy decoding on the target syntax element. While sequentially decoding and parsing each bin of the target syntax element from the input bits in the bitstream, the decoding device can compare the derived bin string with the available bin strings of the syntax element. When the derived bin string matches one of the available bin strings, the value corresponding to the bin string can be derived as the value of the syntax element. If not, the next bit in the bitstream is further parsed and the above process is repeated. Through this process, instead of using the start or end bits of specific information (specific syntax elements) in the bitstream, variable-length bits are used to signal information. This assigns relatively fewer bits to low values, thereby increasing overall coding efficiency.

[0162] The decoding device can perform context-based or bypass-based decoding of individual bins in a bin string from a bitstream based on an entropy coding scheme such as CABAC or CAVLC. The entropy decoding process can be executed by the entropy decoding processor 212 in the entropy decoder 210. The bitstream can include various types of information used for image / video decoding as described above. As mentioned above, the bitstream can be transmitted to the decoding device via a (digital) storage medium or a network.

[0163] In this document, a table including syntax elements (syntax table) can be used to represent information signaling from an encoding device to a decoding device. The order of syntax elements in the table including the syntax elements used in this document can refer to the parsing order of syntax elements from the bitstream. The encoding device can construct and encode the syntax table so that the decoding device parses the syntax elements in the parsing order. The decoding device can parse and decode the syntax elements of the syntax table from the bitstream in the parsing order, and thus obtain the values ​​of the syntax elements.

[0164] General image / video encoding process

[0165] In image / video coding, frames that constitute an image / video can be encoded / decoded sequentially according to the decoding order. The output order of the decoded frames can be set to be different from the decoding order, and based on this, forward prediction and backward prediction can be performed when performing inter-frame prediction.

[0166] Figure 13 This document illustrates an example of a schematic image decoding process to which the embodiments described herein apply. Figure 13 In the above, step S1310 can be referred to... Figure 3The entropy decoder 210 of the described decoding device is executed. Step S1320 can be executed by a prediction unit including an intra-frame prediction unit 265 and an inter-frame prediction unit 260. Step S1330 can be executed by a residual processor including a dequantizer 220 and an inverse transformer 230. Step S1340 can be executed by an adder 235. Step S1350 can be executed by a filter 240. Step S1310 may include the information decoding process described in this document. Step S1320 may include the inter-frame / intra-frame prediction process described in this document. Step S1330 may include the residual processing process described in this document. Step S1340 may include the block / frame reconstruction process described in this document. Step S1350 may include the in-loop filtering process described in this document.

[0167] Reference Figure 13 As shown above Figure 3 The described image decoding process can schematically include the image / video information acquisition process from the bitstream in step S1310 (through decoding), the image reconstruction process in steps S1320 to S1340, and the in-loop filtering process for the reconstructed image in step S1350. The image reconstruction process can be performed based on prediction samples and residual samples obtained through inter-frame / intra-frame prediction in step S1320 and residual processing (dequantization and inverse transform of quantization transform coefficients) in step S1330 as described in this document. The modified reconstructed image can be generated by the in-loop filtering process of the reconstructed image generated by the image reconstruction process. The modified reconstructed image can be output as a decoded image and can be stored in the decoded image buffer or memory 250 of the decoding device for later use as a reference image during inter-frame prediction when decoding the image. In some cases, the in-loop filtering process can be omitted. In this case, the reconstructed image can be output as a decoded image and can be stored in the decoded image buffer or memory 250 of the decoding device for later use as a reference image during inter-frame prediction when decoding the image. As described above, the in-loop filtering process in step S1350 may include a deblocking filtering process, a sample adaptive offset (SAO) process, an adaptive loop filter (ALF) process, and / or a bilateral filter process. Some or all of these may be omitted. Furthermore, one or more of the deblocking filtering process, the sample adaptive offset (SAO) process, the adaptive loop filter (ALF) process, and / or the bilateral filter process may be applied sequentially, or all of them may be applied sequentially. For example, the SAO process may be performed after the deblocking filtering process is applied to the reconstructed image. Alternatively, for example, the deblocking filtering process may be applied to the reconstructed image, and then the ALF process may be performed. This can be performed in the same manner as in the encoding device.

[0168] Figure 14This document illustrates an example of a schematic screen encoding process to which the embodiments described herein apply. Figure 14 In the process, step S1410 can be performed by including the above reference. Figure 2 The prediction unit of the intra-frame prediction unit 185 or inter-frame prediction unit 180 of the described coding apparatus is executed. Step S1420 may be executed by a residual processor including a transformer 120 and / or a quantizer 130. Step S1430 may be executed by an entropy encoder 190. Step S1410 may include the inter-frame / intra-frame prediction process described in this document. Step S1420 may include the residual processing process described in this document. Step S1430 may include the information encoding process described in this document.

[0169] Reference Figure 14 As shown above Figure 2 The described image encoding process can schematically include a process for encoding information for image reconstruction (e.g., prediction information, residual information, and segmentation information) and outputting that information as a bitstream, a process for generating a reconstructed image of the current image, and an optional process for applying in-loop filtering to the reconstructed image. The encoding device can derive (modified) residual samples from the quantized transform coefficients using dequantizer 140 and inverse transformer 150, and can generate a reconstructed image based on the prediction samples and (modified) residual samples as output in step S1410. The generated reconstructed image can be the same as the reconstructed image generated by the decoding device described above. An in-loop filtering process can be performed on the reconstructed image to generate a modified reconstructed image. The modified reconstructed image can be stored in the decoding image buffer or memory 170. Similar to the case in the decoding device, the modified reconstructed image can be used as a reference image later during inter-frame prediction when encoding images. As mentioned above, in some cases, some or all of the in-loop filtering process can be omitted. When the in-loop filtering process is performed, the (in-loop) filtering-related information (parameters) can be encoded by entropy encoder 190 and output as a bitstream. The decoding device can perform the in-loop filtering process in the same way as the encoding device, based on filtering-related information.

[0170] This in-loop filtering process reduces noise generated during image / video encoding (e.g., block artifacts and ringing artifacts), improving both subjective and objective visual quality. Furthermore, since both the encoding and decoding devices perform in-loop filtering, they can derive the same predictions, increasing the reliability of image encoding and reducing the amount of data transmitted for image encoding.

[0171] As described above, the image reconstruction process can be performed in both the decoding and encoding devices. Reconstructed blocks can be generated based on individual blocks using intra-frame prediction / inter-frame prediction, and a reconstructed image including these blocks can be generated. When the current image / slice / patch group is an I-frame / slice / patch group, the blocks included in the current image / slice / patch group can be reconstructed solely based on intra-frame prediction. Furthermore, when the current image / slice / patch group is a P-frame / slice / patch group or a B-frame / slice / patch group, the blocks included in the current image / slice / patch group can be reconstructed based on either intra-frame prediction or inter-frame prediction. In this case, inter-frame prediction can be applied to some blocks in the current image / slice / patch group, and intra-frame prediction can be applied to some remaining blocks. The color components of the image can include luma and chroma components. Unless explicitly limited herein, the methods and implementations presented herein can be applied to both luma and chroma components.

[0172] Examples of encoding levels and structures

[0173] The encoded video / images according to this document can be processed according to, for example, the encoding hierarchy and structure described later.

[0174] Figure 15 This is a diagram illustrating the hierarchical structure of an encoded image. An encoded image can be divided into the Video Coding Layer (VCL), which processes the image and performs its own decoding; a subsystem for transmitting and storing encoded information; and the Network Abstraction Layer (NAL), which exists between the VCL and the subsystems and is responsible for network adaptation functions.

[0175] In VCL, VCL data including compressed image data (slice data) can be generated. Alternatively, parameter sets including information such as Picture Parameter Set (PPS), Sequence Parameter Set (SPS), and Video Parameter Set (VPS) or additional supplemental enhancement information (SEI) messages required for image decoding processing can be generated.

[0176] In NAL, header information (NAL cell header) is added to the raw byte sequence payload (RBSP) generated in VCL to enable the generation of NAL cells. In this document, RBSP refers to the slice data, parameter set, and SEI message generated in VCL. The NAL cell header may include NAL cell type information specified based on the RBSP data included in the NAL cell.

[0177] As shown in the figure, NAL units can be divided into VCL NAL units and non-VCL NAL units based on the RBSP generated in the VCL. VCL NAL units can refer to NAL units that include information about the image (slice data). Non-VCL NAL units can refer to NAL units that include information required for decoding the image (parameter set or SEI message).

[0178] With regard to the data standard header information of the subsystem, VCL NAL units and non-VCL NAL units can be transmitted over a network. For example, NAL units can be transformed into data formats according to predetermined standards such as H.266 / VVC file format, Real-time Transport Protocol (RTP), and Transport Stream (TS), and the resulting data can be transmitted over various networks.

[0179] As mentioned above, regarding NAL cells, the NAL cell type can be specified based on the RBSP data structure included in the NAL cell, and information about the NAL cell type can be stored in the NAL cell header and notified by a signal.

[0180] For example, based on whether NAL units include information about the image (slice data), they can be broadly classified into VCL NAL unit types and non-VCL NAL unit types. VCL NAL unit types can be classified according to the characteristics and type of the image included in the VCL NAL unit, while non-VCL NAL unit types can be classified according to the type of parameter set.

[0181] The following examples list the NAL cell types specified based on the types of the parameter sets included in the non-VCL NAL cell types.

[0182] -APS (Adaptive Parameter Set) NAL Units: NAL unit types including APS.

[0183] -DPS (Decoding Parameter Set) NAL Unit: Includes NAL unit types for DPS.

[0184] -VPS (Video Parameter Set) NAL Unit: Includes the NAL unit type of VPS.

[0185] -SPS (Sequence Parameter Set) NAL Unit: Includes NAL unit types related to SPS.

[0186] -PPS (Picture Parameter Set) NAL Unit: Includes the NAL unit type of PPS.

[0187] The aforementioned NAL unit type can have syntax information for the NAL unit type, and this syntax information can be stored in the NAL unit header and signaled. For example, the syntax information can be nal_unit_type, and the NAL unit type can be specified by the nal_unit_type value.

[0188] A slice header (slice header syntax) may include information / parameters commonly applicable to a slice. APS (APS syntax) or PPS (PPS syntax) may include information / parameters commonly applicable to one or more slices or frames. SPS (SPS syntax) may include information / parameters commonly applicable to one or more sequences. VPS (VPS syntax) may include information / parameters commonly applicable to multiple layers. DPS (DPS syntax) may include information / parameters commonly applicable to the entire video. DPS may include information / parameters related to the concatenation of encoded video sequences (CVS). In this document, the High-Level Syntax (HLS) may include at least one selected from APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, and slice header syntax.

[0189] In this document, the image / video information encoded by the encoding device and signaled to the decoding device in the form of a bitstream may include segmentation-related information within the frame, intra / inter-frame prediction information, residual information, and in-loop filtering information, as well as information included in the slice header, information included in the APS, information included in the PPS, information included in the SPS, and / or information included in the VPS.

[0190] Overview of Intra-Frame Prediction

[0191] The intra-frame prediction performed by the aforementioned encoding and decoding devices will be described in detail below. Intra-frame prediction can refer to the prediction of a block based on reference samples in the frame to which the current block belongs (hereinafter, the current frame).

[0192] This will refer to Figure 16 Description. When applying intra-prediction to the current block 1601, neighboring reference samples to be used for intra-prediction of the current block 1601 can be derived. The neighboring reference samples of the current block may include: a total of 2×nH samples including sample 1611 adjacent to the left boundary of the current block of size nW×nH and sample 1612 adjacent to the lower left; a total of 2×nW samples including sample 1621 adjacent to the upper boundary of the current block and sample 1622 adjacent to the upper right; and a sample 1631 adjacent to the upper left of the current block. Alternatively, the neighboring reference samples of the current block may include multiple columns of upper neighbor samples and multiple columns of left neighbor samples.

[0193] Additionally, the neighboring reference samples of the current block may include: a total of nH samples 1641 adjacent to the right boundary of the current block of size nW×nH; a total of nW samples 1651 adjacent to the bottom boundary of the current block; and a sample 1642 adjacent to the lower right of the current block.

[0194] However, some neighboring reference samples of the current block may not have been decoded or may be unavailable. In this case, the decoding device can construct neighboring reference samples to be used for prediction by replacing unavailable samples with available samples. Alternatively, neighboring reference samples to be used for prediction can be constructed by interpolation of available samples.

[0195] When deriving neighboring reference samples, (i) the predicted sample can be derived based on the average or interpolation of the neighboring reference samples of the current block, or (ii) the predicted sample can be derived based on a reference sample in the neighboring reference samples of the current block that exists in a specific (prediction) direction relative to the predicted sample. Case (i) can be referred to as non-directional mode or non-angular mode, and case (ii) can be referred to as directional mode or angular mode. Alternatively, a predicted sample can be generated by interpolation using a second neighboring sample and a first neighboring sample in the neighboring reference samples that are in the opposite direction to the prediction direction of the intra-prediction mode of the current block based on the predicted sample of the current block. This case can be referred to as Linear Interpolation Intra-Prediction (LIP). Alternatively, a chroma predicted sample can be generated based on a linear model using luminance samples. This case can be referred to as LM mode. Additionally, a temporary predicted sample for the current block can be derived based on filtered neighboring reference samples, and the predicted sample for the current block can be derived by weighted summing the temporary predicted sample with at least one reference sample derived according to the intra-prediction mode (i.e., an unfiltered neighboring reference sample) in the existing neighboring reference samples. This case can be referred to as Position-Related Intra-Prediction (PDPC). Alternatively, the reference sample line with the highest prediction accuracy can be selected from multiple neighboring reference sample lines of the current block to derive the prediction sample using reference samples in the corresponding line located in the prediction direction. In this case, intra-frame prediction coding can be performed by indicating (signaling) the reference sample line used to the decoding device. This can be referred to as multi-reference line (MRL) intra-frame prediction or MRL-based intra-frame prediction. Furthermore, the current block can be divided into vertical or horizontal sub-partitions to perform intra-frame prediction based on the same intra-frame prediction mode, and neighboring reference samples can be derived and used based on each sub-partition. That is, in this case, the intra-frame prediction mode of the current block is applied equivalently to the sub-partitions, and neighboring reference samples are derived and used based on each sub-partition, thereby increasing intra-frame prediction performance in some cases. This prediction method can be referred to as intra-frame sub-partition (ISP) or ISP-based intra-frame prediction. These intra-frame prediction methods can be referred to as intra-frame prediction types, distinguished from intra-frame prediction modes (e.g., DC mode, planar mode, and directional mode). Intra-frame prediction types can be referred to by various terms such as intra-frame prediction schemes or additional intra-frame prediction modes. For example, intra-prediction types (or additional intra-prediction modes) may include at least one selected from the group consisting of LIP, PDPC, MRL, and ISP. General intra-prediction methods other than specific intra-prediction types such as LIP, PDPC, MRL, and ISP may be referred to as normal intra-prediction types. A normal intra-prediction type may refer to a case where no specific intra-prediction type is applied, and prediction can be performed based on the aforementioned intra-prediction modes. Furthermore, post-filtering may be performed on the derived prediction samples when necessary.

[0196] Specifically, the intra-frame prediction process may include an intra-frame prediction mode / type determination step, a neighboring reference sample derivation step, and a prediction sample derivation step based on the intra-frame prediction mode / type. Additionally, a post-filtering step may be performed on the derived prediction samples when necessary.

[0197] In addition to the intra-prediction types mentioned above, affine linear weighted intra-prediction (ALWIP) can be used. ALWIP can be referred to as linear weighted intra-prediction (LWIP), matrix weighted intra-prediction, or matrix-based intra-prediction (MIP). When MIP is applied to the current block, the predicted samples for the current block can be derived by i) using neighboring reference samples after averaging, ii) performing a matrix-vector multiplication process, and further iii) performing a horizontal / vertical interpolation process if necessary. The intra-prediction mode used for MIP may differ from the intra-prediction mode used in LIP, PDPC, MRL, ISP intra-prediction, or normal intra-prediction. The intra-prediction mode used for MIP can be referred to as MIP intra-prediction mode, MIP prediction mode, or MIP mode. For example, different matrices and offsets used in matrix-vector multiplication can be set according to the intra-prediction mode used for MIP. In this paper, the matrix can be referred to as the (MIP) weighted matrix, and the offset can be referred to as the (MIP) offset vector or (MIP) bias vector. Detailed MIP methods will be described later.

[0198] Reference Figure 17 The block reconstruction process based on intra-frame prediction and intra-frame prediction units in the coding apparatus can schematically include, for example, the following: Step S1710 can be performed by the intra-frame prediction unit 185 of the coding apparatus. Step S1720 can be performed by a residual processor including at least one selected from the group consisting of a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, and an inverse transformer 150 of the coding apparatus. Specifically, step S1720 can be performed by the subtractor 115 of the coding apparatus. In step S1730, prediction information can be derived by the intra-frame prediction unit 185 and encoded by the entropy encoder 190. In step S1730, residual information can be derived by the residual processor and encoded by the entropy encoder 190. The residual information is information about the residual samples. The residual information may include information about the quantization transform coefficients of the residual samples. As described above, residual samples can be derived into transform coefficients by the transformer 120 of the encoding device, and transform coefficients can be derived into quantized transform coefficients by the quantizer 130. Information about the quantized transform coefficients can be encoded by the entropy encoder 190 through the residual encoding process.

[0199] In step S1710, the encoding device may perform intra-prediction on the current block. The encoding device derives the intra-prediction mode / type of the current block, derives the neighboring reference samples of the current block, and generates prediction samples in the current block based on the intra-prediction mode / type and the neighboring reference samples. In this document, the processes of determining the intra-prediction mode / type, deriving the neighboring reference samples, and generating the prediction samples may be performed simultaneously, or any one of these processes may be performed before the others. For example, although not shown, the intra-prediction unit 185 of the encoding device may include an intra-prediction mode / type determination unit, a reference sample derivation unit, and a prediction sample derivation unit. The intra-prediction mode / type determination unit may determine the intra-prediction mode / type of the current block, the reference sample derivation unit may derive the neighboring reference samples of the current block, and the prediction sample derivation unit may derive the prediction samples of the current block. Furthermore, when performing the prediction sample filtering process (described later), the intra-prediction unit 185 may also include a prediction sample filter. The encoding device may determine the mode / type among a plurality of intra-prediction modes / types applicable to the current block. The encoding device can compare the RD costs of intra-prediction modes / types and determine the best intra-prediction mode / type for the current block.

[0200] In addition, the encoding device can perform a prediction sample filtering process. Prediction sample filtering can also be called post-filtering. Through the prediction sample filtering process, some or all prediction samples can be filtered. In some cases, the prediction sample filtering process can be omitted.

[0201] In step S1720, the encoding device can generate residual samples for the current block based on the (filtered) prediction samples. The encoding device can compare the prediction samples in the original samples of the current block based on the phase and can derive the residual samples.

[0202] In step S1730, the encoding device can encode image information including information about intra-frame prediction (prediction information) and residual information about residual samples. The prediction information may include intra-frame prediction mode information and intra-frame prediction type information. The encoding device can output the encoded image information as a bitstream. The output bitstream can be transmitted to the decoding device via a storage medium or network.

[0203] Residual information may include residual coding syntax (described later). The encoding device can derive quantization transform coefficients by transforming / quantizing residual samples. Residual information may include information about the quantization transform coefficients.

[0204] Furthermore, as described above, the encoding device can generate a reconstructed frame (including reconstructed samples and reconstructed blocks). To this end, the encoding device can perform dequantization / inverse transform on the quantized transform coefficients and derive (modified) residual samples. The reason for performing dequantization / inverse transform after the transform / quantization of the residual samples is to derive residual samples identical to those derived by the decoding device as described above. The encoding device can generate a reconstructed block, including reconstructed samples of the current block, based on the predicted samples and the (modified) residual samples. Based on the reconstructed blocks, a reconstructed frame of the current frame can be generated. As described above, the in-loop filtering process can be further applied to the reconstructed frame.

[0205] Reference Figure 18 The video / image decoding process based on intra-frame prediction and intra-frame prediction units in a decoding device can schematically include, for example, the following. The decoding device can perform operations corresponding to those performed by the encoding device.

[0206] Steps S1810 to S1830 can be executed by the intra-frame prediction unit 265 of the decoding device. The prediction information in step S1810 and the residual information in step S1840 can be obtained from the bitstream by the entropy decoder 210 of the decoding device. A residual processor, including the dequantizer 220 or the inverse transformer 230 of the decoding device, or both, can derive the residual samples of the current block based on the residual information. Specifically, the dequantizer 220 of the residual processor can perform dequantization based on the quantization transform coefficients derived from the residual information, and can derive the transform coefficients. The inverse transformer 230 of the residual processor can perform an inverse transform on the transform coefficients, and can derive the residual samples of the current block. Step S1850 can be executed by the adder 235 or the reconstructor of the decoding device.

[0207] Specifically, in step S1810, the decoding device can deduce the intra-prediction mode / type of the current block based on the received prediction information (intra-prediction mode / type information). In step S1820, the decoding device can deduce the neighboring reference samples of the current block. In step S1830, the decoding device can generate prediction samples in the current block based on the intra-prediction mode / type and the neighboring reference samples. In this case, the decoding device can perform a prediction sample filtering process. Prediction sample filtering can be referred to as post-filtering. Through the prediction sample filtering process, some or all prediction samples can be filtered. In some cases, the prediction sample filtering process can be omitted.

[0208] The decoding device can generate residual samples for the current block based on the received residual information. In step S1840, the decoding device can generate reconstructed samples for the current block based on the predicted samples and residual samples, and derive a reconstructed block including the reconstructed samples. Based on the reconstructed block, a reconstructed image of the current frame can be generated. As described above, the in-loop filtering process can be further applied to the reconstructed image.

[0209] In this document, although not shown, the intra-prediction unit 265 of the decoding device may include an intra-prediction mode / type determination unit, a reference sample derivation unit, and a prediction sample derivation unit. The intra-prediction mode / type determination unit may determine the intra-prediction mode / type of the current block based on intra-prediction mode / type information obtained from the entropy decoder 210. The reference sample derivation unit may derive neighboring reference samples of the current block. The prediction sample derivation unit may derive prediction samples of the current block. Furthermore, when performing the above-described prediction sample filtering process, the intra-prediction unit 265 may also include a prediction sample filter.

[0210] For example, intra-luma_mpm_flag may be a flag indicating whether the most probable mode (MPM) or a remaining mode is applied to the current block. When an MPM is applied to the current block, the prediction mode information may also include an index indicating one of the intra-luma_mpm_idx prediction mode candidates. Intra-luma_mpm_idx prediction mode candidates may be constructed as an MPM candidate list or an MPM list. Additionally, when an MPM is not applied to the current block, the intra-luma_mpm_remainder prediction mode information may include remaining mode information indicating one of the remaining intra-luma_mpm_remainder prediction modes besides the MPM candidates. The decoding device can determine the intra-luma_mpm_remainder prediction mode for the current block based on the intra-luma_mpm_remainder prediction mode information. A separate MPM list can be constructed for the aforementioned MIP.

[0211] Furthermore, intra-prediction type information can be implemented in various forms. For example, intra-prediction type information may include intra-prediction type index information indicating one of the intra-prediction types. As another example, intra-prediction type information may include reference sample line information (e.g., intra_luma_ref_idx) indicating whether MRL is applied to the current block and which reference sample line is used when MRL is applied to the current block, ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether ISP is applied to the current block, ISP type information indicating the partition type of the subpartition when ISP is applied (e.g., intra_subpartitions_split_flag), flag information indicating whether PDCP is applied, or flag information indicating whether LIP is applied. Additionally, intra-prediction type information may include a MIP flag indicating whether MIP is applied to the current block.

[0212] Intra-prediction mode information and / or intra-prediction type information can be encoded / decoded using the coding methods described in this document. For example, intra-prediction mode information and / or intra-prediction type information can be encoded / decoded using entropy coding based on truncated (rice) binary codes (e.g., CABAC, CAVLC).

[0213] Overview of Adaptive Color Transformation (ACT)

[0214] Adaptive Color Transform (ACT) is a color space transformation (conversion) technique used to remove unnecessary overlap between color components, and it has been used in HEVC Screen Content Extended Edition. It can also be applied to VVC.

[0215] In HEVC Screen Content Extension (HEVC SCC Extension), ACT is used to adaptively transform the prediction residual from the existing color space to the YCgCo color space. One of the two color spaces can be optionally selected by signaling an ACT flag for each transformation basis.

[0216] For example, a first value of the flag (e.g., 1) can indicate that the residuals of the transform basis are encoded in the original color space. A second value of the flag (e.g., 1) can indicate that the residuals of the transform basis are encoded in the YCgCo color space.

[0217] Figure 19 This is a view illustrating an implementation of the decoding process using ACT. Figure 19 In some implementations, motion compensation prediction may correspond to inter-frame prediction in this disclosure.

[0218] like Figure 19 As shown, the reconstructed image (or reconstructed block, reconstructed sample array, reconstructed sample, reconstructed signal) can be generated based on the predicted output value and the residual output value. In this paper, the residual output value can be the inverse transform output value. In this paper, the inverse transform can be the normal inverse transform. In this paper, the normal inverse transform can be the inverse transform based on MTS or the inverse low-frequency non-separable transform (LFNST).

[0219] In this paper, the predicted output value can be a predicted block, a predicted sample array, a predicted sample, or a predicted signal. The residual output value can be a residual block, a residual sample array, a residual sample, or a residual signal.

[0220] For example, in terms of the encoding device, ACT processing can be performed on the residual samples derived from the predicted samples. Alternatively, the output of the ACT processing can be used as input for the normal transform processing. In this paper, the normal transform processing can be an MTS-based transform or an LFNST-based transform.

[0221] Information (parameters) about (inverse)ACT can be generated and encoded by the encoding device and sent to the decoding device in the form of a bit stream.

[0222] The decoding device can obtain, parse, and decode (inverse)ACT-related information (parameters), and can perform inverse ACT based on (inverse)ACT-related information (parameters).

[0223] Based on the inverse ACT, (modified) residual samples (or residual blocks) can be derived. For example, (transform) coefficients can be derived by applying dequantization to the quantization (transform) coefficients. Alternatively, residual samples can be derived by performing an inverse transform on the (transform) coefficients. Furthermore, (modified) residual samples can be obtained by applying the inverse ACT to the residual samples. Information (parameters) regarding the (inverse) ACT will be described in detail later.

[0224] In implementations, the core transformation function used in HEVC can be used as the core transformation function (transformation kernel) for color space transformation. For example, matrices for forward and backward transformations, as shown in the following equation, can be used.

[0225] [Formula 1]

[0226] [Equation 2]

[0227] In this paper, C0, C1, and C2 can correspond to G, B, and R, respectively. G represents the green component, B represents the blue component, and R represents the red component. Additionally, C0', C1', and C2' can correspond to Y, Cg, and Co, respectively. Y represents luminance, Cg represents green chromaticity, and Co represents the orange chromaticity component.

[0228] In addition, to compensate for the change in dynamic range of the residuals before and after color transformation, QP adjustment can be applied to the transformation residuals at (-5, -5, -3). Details of QP adjustment will be described later.

[0229] Furthermore, in the encoding and decoding processes according to the implementation method, when ACT is applicable, the following limitations may be applied.

[0230] - In the case of dual-tree encoding / decoding, disable ACT. For example, ACT can be applied only to single-tree encoding / decoding.

[0231] - ACT can be disabled when applying ISP encoding and decoding.

[0232] - For chroma blocks that have applied BDPCM, ACT can be disabled. ACT can only be enabled for luma blocks that have applied BDPCM.

[0233] - CCLM can be disabled when ACT can be applied.

[0234] Figure 20 This is a view illustrating an implementation of a sequence parameter set syntax table that uses signals to notify grammatical elements related to ACT.

[0235] Figures 21 to 27 It is a view that continuously shows an implementation of a syntax table that uses signals to notify the grammatical elements related to ACT.

[0236] like Figure 20 As shown, the ACT enabling flag, which indicates whether ACT is enabled in the decoding process, can be sps_act_enabled_flag 2010.

[0237] The first value of sps_act_enabled_flag (e.g., 0) can indicate that ACT is not used, and there is no flag provided in the coding base syntax to indicate whether ACT is applied on the coding base (cu_act_enabled_flag 2110, 2710).

[0238] The second value of sps_act_enabled_flag (e.g., 1) can indicate that ACT can be used, and cu_act_enabled_flag can be provided in the syntax of the coding base.

[0239] When sps_act_enabled_flag is not available in the bitstream, the value of sps_act_enabled_flag can be deduced to be the first value (e.g., 0).

[0240] In addition, such as Figure 21 As shown, the ACT flag, which indicates whether the residual of the current encoding basis is encoded in the YCgCo color space, can be cu_act_enabled_flag 2110, 2710.

[0241] The first value of cu_act_enabled_flag (e.g., 0) can indicate that the residual of the current encoding base is encoded in the original color space. The second value of cu_act_enabled_flag (e.g., 1) can indicate that the residual of the current encoding base is encoded in the YCgCo color space.

[0242] When the cu_act_enabled_flag is not provided in the bitstream, this flag can be deduced to a first value (e.g., 0). In this article, the original color space can be the RGB color space.

[0243] QP derivation method based on the transformation of ACT QP offset

[0244] In the implementation, the quantization parameter derivation and Qp update processes in the scaling of the transform coefficients can be performed as follows. For example, the quantization parameter derivation can be performed using the following parameters.

[0245] - Luminosity coordinates (xCb, yCb), relative to the top-left luminosity sample of the current frame, indicating the relative coordinates of the top-left luminosity sample of the current coded block. - The variable cbWidth indicates the width of the current coded block based on each luminance sample. - The variable cbHeight indicates the height of the current coding block based on each luminance sample. - The variable treeType indicates whether a single tree (SINGLE_TREE) or a dual tree is used to split the current coding tree node, and when a dual tree is used, it indicates whether the dual tree is a luma component dual tree (DAUL_TREE_LUMA) or a chroma component dual tree (DAUL_TREE_CHROMA).

[0246] In this process, the luminance quantization parameter Qp'Y, the chromaticity quantization parameters Qp'Cb, Qp'Cr, and Qp'CbCr can be derived.

[0247] The variable brightness position (xQg, yQg) indicates the position of the top-left brightness sample of the current quantization group, corresponding to the top-left sample of the current image. In this paper, the horizontal position xQg and the vertical position yQg can be set to equal the values ​​of the variables CuQgTopLeftX and CuQgTopLeftY, respectively. Figure 28 In the encoding tree syntax shown, variables CuQgTopLeftX and CuQgTopLeftY can be defined as predetermined values.

[0248] In this paper, the current quantization group can be a quadrilateral region within a coding tree block, and they can share the same qP. Y_PRED The value. Its width and height can be equal to the width and height of the coding tree node, where the top-left luminance sample position is assigned to each of CuQgTopLeftX and CuQgTopLeftY.

[0249] When treeType is SINGLE_TREE or DUAL_TREE_LUMA, the predicted value of the luminance quantization parameter qP is... Y_PRED It can be derived as follows:

[0250] 1. Variable qP Y_PRED The following derivation can be made.

[0251] (Condition 1) q is true if any one of the following conditions is true. PY_PREDThe value can be set to match SliceQp Y Same value (in this article, SliceQp) Y Indicate the quantization parameter Qp for all slices in the image. Y The initial value, which can be obtained from the bitstream. Alternatively, qP Y_PRED The value can be set to the luminance quantization parameter Qp based on the last luminance code of the quantization group immediately preceding the decoding order. Y The value of .

[0252] -(Condition 1-1) When the current quantization group is the first quantization group in the slice

[0253] -(Condition 1-2) When the current quantization group is the first quantization group in the block.

[0254] - (Conditions 1-3) When the current quantization group is the first quantization group in the CTB line of the concatenation and a scheduled synchronization occurs (e.g., when entropy_coding_sync_enabled_flag has a value of 1).

[0255] 2. Variable qP Y_A The value of can be derived as follows.

[0256] (Condition 2) qP is true when at least one of the following conditions is true. Y_A The value can be set to qP Y_PRED The value of qP. Alternatively, qP Y_A The value can be set to the luminance quantization parameter Qp, which is the coding basis of the luminance coding block covering the luminance sample location (xQg-1, yQg). Y The value of .

[0257] - (Condition 2-1) The block identified by the sample position (xQg-1, yQg) is not a usable neighboring block relative to the block identified by the sample position (xCb, yCb). - (Condition 2-2) When the CTB of the luminance-coded block including the luminance sample position (xQg-1, yQg) is different from the CTB of the current luminance-coded block including the luminance sample position (xCb, yCb), for example, when all of the following conditions are true. -(Condition 2-2-1) The value of (xQg-1)>>CtbLog2SizeY is different from the value of (xCb)>>CtbLog2SizeY. -(Condition 2-2-2) The value of (yQg)>>CtbLog2SizeY is different from the value of (yCb)>>CtbLog2SizeY. 3. Variable qP Y_B The value of can be derived as follows.

[0258] (Condition 3) qP is true when at least one of the following conditions is true. Y_B The value can be set to qP Y_PRED The value of qP. Alternatively, qP Y_B The value can be set to the luminance quantization parameter Qp, which is the coding basis of the luminance coding block covering the luminance sample location (xQg, yQg-1). Y The value of .

[0259] - (Condition 3-1) When the block identified by the sample position (xCb, yCb) is not a usable neighboring block, relative to the block identified by the sample position (xQg, yQg-1), - (Condition 3-2) When the CTB of the luminance-coded block including the luminance sample location (xQg, yQg-1) is different from the CTB of the current luminance-coded block including the luminance sample location (xCb, yCb), for example, when all of the following conditions are true. -(Condition 3-2-1) The value of (xQg)>>CtbLog2SizeY is different from the value of (xCb)>>CtbLog2SizeY. -(Condition 3-2-2) (yQg-1)>>CtbLog2SizeY has a different value than (yCb)>>CtbLog2SizeY. 4. Predicted value of brightness quantization parameter qP Y_PRED The following derivation can be made.

[0260] qP is true when all of the following conditions are true. Y_PRED The luminance quantization parameter Qp can be set as the encoding basis for a luminance coding block that covers the luminance sample positions (xQg, yQg-1). Y .

[0261] -(Condition 3-1) The block identified by the sample position (xQg, yQg-1) is a usable neighboring block relative to the block identified by the sample position (xCb, yCb).

[0262] - When the current quantization group is the first quantization group in the CTB line of the block.

[0263] Furthermore, when all conditions are false, qP Y_PRED It can be derived as shown in the following formula.

[0264] [Formula 3]

[0265] variable Qp Y It can be derived as shown in the following formula.

[0266] [Formula 4]

[0267] In this paper, CuQpDeltaVal indicates the difference between the luminance quantization parameter underlying the encoding and its predicted value. Its value can be obtained from the bitstream. QpBdOffset indicates the range offset of the luminance and chrominance quantization parameters. QpBdOffset can be preset to a predetermined constant or obtained from the bitstream. For example, QpBdOffset can be calculated by multiplying the predetermined constant by the value of the syntax element indicating the bit depth of the luminance or chrominance sample. The luminance quantization parameter Qp' Y It can be derived as shown in the following formula.

[0268] [Formula 5]

[0269] Qp' Y =Qp Y +QpBdOffset

[0270] The following processing can be performed when the value of the variable ChromaArrayType, which indicates the type of the chroma array, is not the first value (e.g., 0) and treeType is SINGLE_TREE or DUAL_TREE_CHROMA.

[0271] - When the value of treeType is DUAL_TREE_CHROMA, the variable Qp Y The value can be set to the luminance quantization parameter Qp based on the luminance coding at the luminance sample location (xCb+cbWidth / 2, yCb+cbHeight / 2). Y Same value.

[0272] -Variable qP Cb qP Cr and qP CbCr It can be derived as shown in the following formula.

[0273] [Formula 6]

[0274] The colorimetric parameters Qp' of the Cb and Cr components Cb and Qp' Cr and the colorimetric parameter Qp' of the joint Cb-Cr encoding CbCr It can be derived as shown in the following formula.

[0275] [Formula 7]

[0276] In the above formula, pps_cb_qp_offset and pps_cr_qp_offset are used to derive Qp'. Cb and Qp'Cr The offset, and can be obtained from the bitstream of the frame parameter set. `slice_cb_qp_offset` and `slice_cr_qp_offset` are used to derive Qp'. Cb and Qp' Cr The offset, and can be obtained from the bitstream in the slice header. CuQpOffset Cb and CuQpOffset Cr It is used to derive Qp' Cb and Qp' Cr The offset can be obtained from the bitstream of the transform basis.

[0277] Alternatively, for example, the dequantization of the transform coefficients can be performed using the following parameters.

[0278] - Luminosity coordinates (xTbY, yTbY), relative to the top-left luminance sample of the current image, indicating the relative coordinates of the top-left sample of the current luminance transform block.

[0279] - The variable nTbW indicates the width of the transform block.

[0280] - Variable nTbH indicates the height of the transform block.

[0281] - The variable cIdx indicates the color components of the current block.

[0282] The output of this process can be an array d of scaled transform coefficients. In this paper, the size of array d can be (nTbW)×(nTbH). The individual elements constituting this array can be labeled d[x][y].

[0283] Therefore, the quantization parameter qP can be derived as follows. When cIdx has a value of 0, qP can be derived as shown in the following equation.

[0284] [Formula 8]

[0285] qP=Qp' Y

[0286] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, it can be derived as shown in the following formula.

[0287] [Formula 9]

[0288] qP=Qp' CbCr

[0289] Alternatively, when cIdx has a value of 1, qP can be derived as shown in the following equation.

[0290] [Formula 10]

[0291] qP=Qp' Cb

[0292] Alternatively, when cIdx has a value of 2, qP can be derived as shown in the following equation.

[0293] [Equation 11]

[0294] qP=Qp' Cr

[0295] Subsequently, the quantization parameter qP can be updated as follows. Additionally, the variables rectNonTsFlag and bdShift can be derived as follows. For example, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0 (e.g., when the current transform block is not skipped), qP can be derived as shown in the following equation.

[0296] [Equation 12]

[0297] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1 (e.g., skipping the transformation of the current transform block), qP can be derived as shown in the following equation.

[0298] [Equation 13]

[0299] In this paper, QpPrimeTsMin indicates the minimum quantization parameter value allowed when a transform skip mode is applied. This can be determined as a predetermined constant or derived from the syntax elements of the bitstream associated with it.

[0300] In this article, the suffixes Y, Cb, and Cr can represent the G, B, and R color components in the RGB color model or the Y, Cg, and Co color components in the YCgCo color model.

[0301] Overview of Block Differential Pulse Code Modulation (BDPCM)

[0302] The image encoding apparatus and image decoding apparatus according to the embodiments can perform differential encoding of the residual signal. For example, the image encoding apparatus can encode the residual signal by subtracting the prediction signal from the residual signal of the current block, and the image decoding apparatus can decode the residual signal by adding the prediction signal to the residual signal of the current block. The image encoding apparatus and image decoding apparatus according to the embodiments can perform differential encoding of the residual signal by applying the following BDPCM.

[0303] The BDPCM according to this disclosure can be performed in the quantization residual domain. The quantization residual domain may include the quantization residual signal (or quantization residual coefficients), and when BDPCM is applied, the transformation of the quantization residual signal can be skipped. For example, when BDPCM is applied, the transformation of the residual signal can be skipped and quantization can be performed. Alternatively, the quantization residual domain may include quantization transform coefficients.

[0304] In implementations using BDPCM, the image coding device can derive the residual block of the current block predicted in intra-frame prediction mode and quantize the residual block, thereby deriving the residual block. When performing differential coding mode on the residual signal for the current block, the image coding device can perform differential coding on the residual block to derive the modified residual block. Additionally, the image coding device can encode differential coding mode information of the specified residual signal and the modified residual block to generate a bitstream.

[0305] More specifically, when BDPCM is applied to the current block, a predicted block (prediction block) including predicted samples of the current block can be generated through intra-frame prediction. In this case, the intra-frame prediction mode used to perform intra-frame prediction can be signaled via a bitstream and can be derived based on the prediction direction of BDPCM. Furthermore, in this case, the intra-frame prediction mode can be determined to be either a vertical prediction direction mode or a horizontal prediction direction mode. For example, when the prediction direction of BDPCM is horizontal, the intra-frame prediction mode can be determined to be a horizontal prediction direction mode, and the prediction block of the current block can be generated through horizontal intra-frame prediction. Alternatively, when the prediction direction of BDPCM is vertical, the intra-frame prediction mode can be determined to be a vertical prediction direction mode, and the prediction block of the current block can be generated through vertical intra-frame prediction. When horizontal intra-frame prediction is applied, the values ​​of the pixels adjacent to the left side of the current block can be determined as predicted sample values ​​of the samples included in the corresponding row of the current block. When vertical intra-frame prediction is applied, the values ​​of the pixels adjacent to the top of the current block can be determined as predicted sample values ​​of the samples included in the corresponding column of the current block. When BDPCM is applied to the current block, the method for generating the prediction block of the current block can be performed equally in the image encoding device and the image decoding device.

[0306] When applying BDPCM to the current block, the image coding device can generate a residual block that includes residual samples of the current block by subtracting the prediction samples from the current block. The image coding device can quantize the residual block and then encode the difference (or increment) between the quantized residual samples and the predictors of the quantized residual samples. The image decoding device can generate a quantized residual block of the current block by obtaining the quantized residual samples of the current block based on the predictors and the difference reconstructed from the bitstream. Subsequently, the image decoding device can dequantize the quantized residual block and then add it to the prediction block to reconstruct the current block.

[0307] Figure 29 This is a view illustrating a method for encoding residual samples of BDPCM according to this disclosure. Figure 29 The residual block can be generated by subtracting the predicted block from the current block in the image coding device. Figure 29 The quantized residual blocks can be generated by quantizing the residual blocks using an image coding device. Figure 29 In the middle, r i, j Specifies the value of the residual sample at coordinate (i, j) in the current block. When the size of the current block is M×N, the value i can be 0 to M-1 (inclusive). Similarly, the value j can be 0 to N-1 (inclusive). For example, the residual can refer to the difference between the original block and the predicted block. For example, r i, j This can be derived by subtracting the predicted sample value from the original sample value at coordinate (i, j) in the current block. For example, r i, j This can be the prediction residual after performing horizontal or vertical intra-frame prediction using samples that were not filtered from the top or left boundary. In horizontal intra-frame prediction, the values ​​of the left neighboring pixels are copied along the lines that intersect the prediction block. In vertical intra-frame prediction, the top neighboring row is copied to a separate row of the prediction block.

[0308] exist Figure 29 In, Q(r) i, j ) refers to the value of the quantized residual sample at coordinate (i, j) in the current block. For example, Q(r i, j ) can refer to r i, j The quantized value.

[0309] right Figure 29 The quantized residual samples are used to perform BDPCM predictions, and a modified quantized residual block R' of size M×N can be generated, which includes the modified quantized residual sample r'.

[0310] When the prediction direction of BDPCM is horizontal, the modified quantized residual sample value r' of the coordinate (i, j) in the current block. i, j It can be calculated as shown in the following formula.

[0311] [Formula 14]

[0312] As shown in Equation 14, when the prediction direction of BDPCM is horizontal, the value of the quantized residual sample Q(r) is... 0, j The value r' assigned as is to the coordinate (0, j) 0, j The values ​​of other coordinates (i, j) r' i, j The value of the quantized residual sample Q(r) can be derived as the coordinate (i, j). i, j The value of the quantized residual sample Q(r) with coordinates (i-1, j) i-1, j The difference between (i, j). That is, the value Q(r) that replaces the quantized residual sample for coordinate (i, j). i, j Encode using the value Q(r) of the quantized residual sample at coordinate (i-1, j). i-1, j The difference calculated as the predicted value is derived as the modified quantized residual sample value r'. i, j Then for the value r' i, j Encode it.

[0313] When the prediction direction of BDPCM is vertical, the modified quantized residual sample value (r') at coordinate (i, j) in the current block. i, j It can be calculated as shown in the following formula.

[0314] [Formula 15]

[0315] As shown in Equation 15, when the prediction direction of BDPCM is vertical, the value of the quantized residual sample Q(r) is... i, 0 The value r' assigned to coordinate (i, 0) as is i, 0 The values ​​of other coordinates (i, j) r' i, j The value of the quantized residual sample Q(r) can be derived as the coordinate (i, j). i, j The value of the quantized residual sample Q(r) with coordinates (i, j-1) i, j-1 The difference between (i, j). That is, the value Q(r) that replaces the quantized residual sample for coordinate (i, j). i, j Encode using the value Q(r) of the quantized residual sample at coordinate (i, j-1). i, j-1 The difference calculated as the predicted value is derived as the modified quantized residual sample value r'. i, j Then for the value r' i, j Encode it.

[0316] As mentioned above, the process of modifying the current quantized residual sample value using nearby quantized residual sample values ​​as predicted values ​​can be called BDPCM prediction.

[0317] Finally, the image encoding device can encode the modified quantization residual block, including the modified quantization residual samples, and can send the resulting block to the image decoding device. In this paper, as described above, the transformation of the modified quantization residual block is not performed.

[0318] Figure 30 This is a view showing the modified quantization residual block generated by performing BDPCM according to this disclosure.

[0319] exist Figure 30 In the diagram, the horizontal BDPCM shows the modified quantization residual block generated according to Equation 14 when the prediction direction of the BDPCM is horizontal. Similarly, the vertical BDPCM shows the modified quantization residual block generated according to Equation 15 when the prediction direction of the BDPCM is vertical.

[0320] Figure 31 This is a flowchart illustrating the process of encoding the current block in an image encoding device by applying BDPCM.

[0321] First, when the current block, which is the target block for encoding, is input in step S3110, prediction can be performed on the current block in step S3120 to generate a prediction block. The prediction block in step S3120 can be an intra-frame prediction block, and the intra-frame prediction mode can be determined as described above. Based on the prediction block generated in step S3120, a residual block of the current block can be generated in step S3130. For example, the image coding device can generate a residual block (the value of the residual sample) by subtracting the prediction block (the value of the predicted sample) from the current block (the value of the original sample). For example, by executing step S3130, a residual block (the value of the residual sample) can be generated. Figure 29 The residual block generated in step S3130 can be quantized in step S3140 to generate a quantized residual block, and BDPCM prediction can be performed on the quantized residual block in step S3150. The quantized residual block generated as a result of performing step S3140 can be... Figure 29 The quantized residual block. This serves as the result of the BDPCM prediction in step S3150. Figure 30 The modified quantization residual block can be generated based on the prediction direction. Since the BDPCM prediction in step S3150 has already referenced... Figures 29 to 30 The description has already been provided, so its detailed description will be omitted. Subsequently, the image encoding device can encode the modified quantization residual block in step S3160 to generate a bitstream. In this document, the transformation of the modified quantization residual block can be skipped.

[0322] Reference Figures 29 to 31 The BDPCM operation in the described image encoding device can be reversed and performed by the image decoding device.

[0323] Figure 32This is a flowchart illustrating the process of reconstructing the current block in an image decoding device by applying BDPCM.

[0324] In step S3210, the image decoding device can obtain the information (image information) required to reconstruct the current block from the bitstream. The information required to reconstruct the current block may include prediction information about the current block (prediction information) and residual information about the current block (residual information). In step S3220, the image decoding device can perform prediction on the current block based on the information about the current block and can generate a prediction block. The prediction of the current block can be intra-frame prediction, the details of which are described above. Figure 31 The descriptions are the same. In Figure 32 The diagram shows that step S3220, which generates the prediction block of the current block, is performed before steps S3230 to S3250, which generate the residual block of the current block. However, no restrictions are imposed on this step. The prediction block of the current block can be generated after the residual block of the current block is generated. Alternatively, the residual block and the prediction block of the current block can be generated simultaneously.

[0325] In step S3230, the image decoding device can generate a residual block for the current block by parsing the residual information of the current block from the bitstream. The residual block generated in step S3230 can be... Figure 30 The modified quantization residual block is shown.

[0326] The image decoding device can perform the following steps in step S3240: Figure 30 The modified quantized residual block is used to perform BDPCM prediction to generate Figure 29 The quantized residual block. The BDPCM prediction in step S3240 is from... Figure 30 Modified quantization residual block generation Figure 29 The process of quantizing the residual block corresponds to the inverse processing of step S3150 performed by the image encoding device. For example, when the differential coding mode information (e.g., bdpcm_flag) obtained from the bitstream indicates a differential coding mode for differential coding of residual coefficients when BDPCM is applied, the image decoding device performs differential coding on the residual block to derive the modified residual block. Using the residual coefficient to be modified and the predicted residual coefficient, the image decoding device can modify at least one residual coefficient in the residual block that needs to be modified. The predicted residual coefficient can be determined based on the prediction direction indicated by the differential coding direction information (e.g., bdpcm_dir_flag) obtained from the bitstream. The differential coding direction information can indicate a vertical or horizontal direction. The image decoding device can assign the value obtained by adding the residual coefficient to be modified and the predicted residual coefficient to the position of the residual coefficient to be modified. In this document, the predicted residual coefficient can be a coefficient that immediately precedes and is adjacent to the residual coefficient to be modified according to the prediction direction.

[0327] The BDPCM prediction in step S3240 performed by the image decoding device will be described in more detail below. The decoding device can calculate the quantized residual sample Q(r) by reversing the calculation performed by the encoding device. i, j For example, when the prediction direction of BDPCM is horizontal, the image decoding device can use Equation 16 to generate a quantization residual block from the modified quantization residual block.

[0328] [Formula 16]

[0329] As defined in Equation 16, the value of the quantized residual sample Q(r) at coordinate (i, j) is... i, j It can be calculated by adding the values ​​of the modified quantized residual samples from coordinate (0,j) to coordinate (i,j).

[0330] Alternatively, by using Equation 17 instead of Equation 16, the value of the quantized residual sample Q(r) at coordinate (i, j) can be calculated. i, j ).

[0331] [Equation 17]

[0332] Equation 17 is the inverse of Equation 14. According to Equation 17, the value of the quantized residual sample at coordinate (0,j) is Q(r). 0, j The value r' of the modified quantized residual sample at coordinate (0, j) is derived. 0, j Q(r) for other coordinates (i, j) i, j The value r' of the modified quantized residual sample at coordinates (i, j) is derived. i, j The value of the quantized residual sample Q(r) at coordinate (i-1, j) i-1, j The sum of ) . That is, the value Q(r) of the quantized residual sample using coordinates (i-1, j). i-1, j The difference r' will be used as the predicted value. i, j Add them together to derive the quantized residual sample value Q(r). i, j ).

[0333] When the prediction direction of BDPCM is vertical, the image decoding device can use Equation 18 to generate a quantization residual block from the modified quantization residual block.

[0334] [Formula 18]

[0335] As defined in Equation 18, the value of the quantized residual sample Q(r) at coordinate (i, j) is... i, jIt can be calculated by adding the values ​​of the modified quantized residual samples from coordinate (i,0) to coordinate (i,j).

[0336] Alternatively, by using Equation 19 instead of Equation 18, the value of the quantized residual sample Q(r) at coordinate (i, j) can be calculated. i, j ).

[0337] [Formula 19]

[0338] Equation 19 is the inverse of Equation 15. According to Equation 19, the value of the quantized residual sample at coordinate (i, 0) is Q(r). i, 0 The value r' of the modified quantized residual sample at coordinate (i, 0) is derived. i, 0 Q(r) for other coordinates (i, j) i, j The value r' of the modified quantized residual sample at coordinates (i, j) is derived. i, j The value of the quantized residual sample Q(r) at coordinate (i, j-1) i, j-1 The sum of ) . That is, the value Q(r) of the quantized residual sample using coordinates (i, j-1). i, j-1 The difference r' will be used as the predicted value. i, j Add them together to derive the quantized residual sample value Q(r). i, j ).

[0339] When a quantized residual block consisting of quantized residual samples is generated by performing step S3240 according to the method described above, the image decoding device performs dequantization on the quantized residual block in step S3250 to generate the residual block of the current block. When BDPCM is applied, the transformation of the current block is skipped as described above. Therefore, the inverse transformation of the dequantized residual block can be skipped.

[0340] Subsequently, the image decoding device can reconstruct the current block in step S3260 based on the prediction block generated in step S3220 and the residual block generated in step S3250. For example, the image decoding device can reconstruct the current block (the value of the reconstructed sample) by adding the prediction block (the value of the predicted sample) and the residual block (the value of the residual sample). For example, this can be achieved by adding the dequantized sample Q. -1 (Q(r i,j The reconstructed sample value is generated by adding it to the intra-block prediction value. The differential coding mode information of BDPCM can be signaled via the bitstream to indicate whether it is applied to the current block. Additionally, when BDPCM is applied to the current block, the differential coding direction information indicating the prediction direction of BDPCM can be signaled via the bitstream. When BDPCM is not applied to the current block, the differential coding direction information does not need to be signaled.

[0341] Figures 33 to 35 This is a schematic view illustrating the syntax of using signals to communicate information about BDPCM.

[0342] Figure 33 This is a view illustrating the syntax of the sequence parameter set according to an implementation of signaling BDPCM information. In the implementation, all SPS RBSPs included in at least one access unit (AU) having a value of 0 as a Temporal ID (TemporalId) or provided by external means can be set to be used before being referenced in the decoding process. Additionally, SPS NAL units including SPSRBSPs can be set to have the same nuh_layer_id as the PPS NAL unit of the reference SPS NAL unit. In CVS, all SPS NAL units with a specific sps_seq_parameter_set_id value can be set to have the same content. Figure 33 The seq_parameter_set_rbsp() syntax exposes the aforementioned sps_transform_skip_enable_flag and the sps_bdpcm_enabled_flag, which will be described later.

[0343] The syntax element `sps_bdpcm_enabled_flag` indicates whether `intra_bdpcm_flag` is provided in the CU syntax for intra-coding units. For example, a first value of `sps_bdpcm_enabled_flag` (e.g., 0) indicates that `intra_bdpcm_flag` is not provided in the CU syntax for intra-coding units. A second value of `sps_bdpcm_enabled_flag` (e.g., 1) indicates that `intra_bdpcm_flag` is provided in the CU syntax for intra-coding units. Furthermore, when `sps_bdpcm_enabled_flag` is not provided, its value can be set to the first value (e.g., 0).

[0344] Figure 34 This is a view illustrating an implementation of a syntax for signaling whether constraints on BDPCM are applied. In this implementation, predetermined constraints in the encoding / decoding process can be signaled using the `general_constraint_info()` syntax. Figure 34The syntax allows for signaling the `no_bdpcm_constraint_flag` element, which indicates whether the value of `sps_bdpcm_enabled_flag` should be set to 0. For example, a first value for `no_bdpcm_constraint_flag` (e.g., 0) can indicate whether this constraint is applied. When the value of `no_bdpcm_constraint_flag` is a second value (e.g., 1), the value of `sps_bdpcm_enabled_flag` can be forced to the first value (e.g., 0).

[0345] Figure 35 This is a view illustrating an implementation of the coding_unit() syntax that uses signals to notify information about the BDPCM of the coding unit. For example... Figure 35 As shown, the coding_unit() syntax can be used to signal the syntax elements intra_bdpcm_flag and intra_bdpcm_dir_flag. The syntax element intra_bdpcm_flag can indicate whether BDPCM should be applied to the current luminance coding block located at (x0, y0).

[0346] For example, a first value of intra_bdpcm_flag (e.g., 0) can indicate that BDPCM is not applied to the current luma coding block. A second value of intra_bdpcm_flag (e.g., 1) can indicate that BDPCM is applied to the current luma coding block. By indicating the application of BDPCM, intra_bdpcm_flag can indicate whether to skip the transform and whether intra-luma prediction mode is performed by intra_bdpcm_dir_flag (described later).

[0347] Furthermore, for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1, the value of the above variable BdpcmFlag[x][y] can be set to the value of intra_bdpcm_flag.

[0348] The syntax element `intra_bdpcm_dir_flag` can indicate the prediction direction of BDPCM. For example, the first value of `intra_bdpcm_dir_flag` (e.g., 0) can indicate that the BDPCM prediction direction is horizontal. The second value of `intra_bdpcm_dir_flag` (e.g., 1) can indicate that the BDPCM prediction direction is vertical.

[0349] Furthermore, for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1, the value of the variable BdpcmDir[x][y] can be set to the value of intra_bdpcm_dir_flag.

[0350] Intra-frame prediction of chroma blocks

[0351] When performing intra-frame prediction on the current block, prediction can be performed on both the luma component block (luma block) and the chroma component block (chroma block). In this case, the intra-frame prediction mode for the chroma block can be set separately from the intra-frame prediction mode for the luma block.

[0352] For example, the intra-prediction mode of a chroma block can be indicated based on intra-chroma prediction mode information. Intra-chroma prediction mode information can be signaled using the syntax element `intra_chroma_pred_mode`. For example, intra-chroma prediction mode information can indicate one of the following: planar mode, DC mode, vertical mode, horizontal mode, derivation mode (DM), and cross-component linear model (CCLM) mode. In this paper, planar mode can refer to intra-prediction mode 0, DC mode can refer to intra-prediction mode 1, vertical mode can refer to intra-prediction mode 26, and horizontal mode can refer to intra-prediction mode 10. DM can also be called direct mode. CCLM can also be called linear model (LM). CCLM mode can include any of the following: L_CCLM, T_CCLM, and LT_CCLM.

[0353] Furthermore, DM and CCLM are subordinate intra-prediction modes that use information from the luma block to predict the chroma block. DM can refer to a mode that applies the same intra-prediction mode as the luma component's intra-prediction mode as the chroma component's intra-prediction mode. Conversely, CCLM can refer to an intra-prediction mode where, during the generation of the chroma block's prediction block, the reconstructed samples of the luma block are undersampled, and the samples derived by applying CCLM parameters α and β to the undersampled samples are used as the prediction samples for the chroma block.

[0354] Overview of Cross-Component Linear Model (CCLM) Modes

[0355] As described above, the CCLM mode can be applied to chroma blocks. The CCLM mode is an intra-frame prediction mode that uses the correlation between luma blocks and their corresponding chroma blocks, and is performed by deriving a linear model based on neighboring samples of the luma blocks and the chroma blocks. Alternatively, predicted samples of the chroma blocks can be derived based on the derived linear model and the reconstructed samples of the luma blocks.

[0356] Specifically, when applying the CCLM mode to the current chroma block, the parameters of the linear model can be derived based on the neighbor samples of the intra-prediction used for the current chroma block and the neighbor samples of the intra-prediction used for the current luma block. For example, the linear model of CCLM can be represented based on the following equation.

[0357] [Formula 20]

[0358] In this article, pred c (i,j) can refer to the predicted sample of the coordinates (i,j) of the current chroma block in the current CU. L '(i,j) can refer to the reconstructed sample of the coordinates (i,j) of the current luma block in the CU. For example, rec L '(i,j) can refer to the downsampled reconstructed sample of the current luma block. The linear model coefficients α and β can be provided by signaling or derived from neighboring samples.

[0359] Joint encoding of residuals (joint CbCr)

[0360] In the encoding / decoding process according to the implementation, chroma residuals can be encoded / decoded together. This can be referred to as joint encoding of residuals or joint CbCr. Whether the joint encoding mode of CbCr is applied (enabled) can be signaled by the joint encoding mode signaling flag tu_joint_cbcr_residual_flag at the transform base level. Additionally, the selected encoding mode can be derived from the chroma CBF. The flag tu_joint_cbcr_residual_flag can exist when at least one chroma CBF of the transform base has a value of 1. The chroma QP offset value indicates the difference between the general chroma QP offset value signaled for the regular chroma residual encoding mode and the chroma QP offset value for the CbCr joint encoding mode. The chroma QP offset value can be signaled via PPS or slice header. This QP offset value can be used to drive the chroma QP values ​​of blocks using the joint chroma residual encoding mode.

[0361] When Mode 2, which is the corresponding joint chroma coding mode in the table below, is enabled for the transform basis, its chroma QP offset can be added to the target luminance-derived chroma QP (the applied luminance-derived chroma QP) while performing quantization and decoding of the transform basis.

[0362] For other modes (similar to modes 1 and 3 in the table below), the chromaticity QP can be derived in the same way as for general Cb or Cr blocks. This process of reconstructing the chromaticity residuals (resCb and resCr) from the transform blocks can be selected according to the table below. When the current mode is enabled, a single joint chromaticity residual block (resJointC[x][y] in the table below) is signaled, and information such as tu_cbf_cb, tu_cbf_cr, and CSign (sign values ​​exposed in the slice header) can be considered to derive the residual block resCb for Cb and the residual block resCr for Cr.

[0363] In the encoding device, the joint chroma components can be derived as follows. Based on the joint encoding mode, resJointC{1, 2} can be generated in the following order. In mode 2 (with reconstructed Cb=C, Cr=CSign)... In the case of individual residuals of C, the joint residuals can be determined according to the following formula.

[0364] [Equation 21]

[0365] resJointC[x][y]=(resCb[x][y]+CSign resCr[x][y]) / 2.

[0366] Alternatively, in mode 1 (with reconstruction Cb=C, Cr=(CSign) In the case of a single residual of C) / 2, the joint residual can be determined according to the following formula.

[0367] [Equation 22]

[0368] resJointC[x][y]=(4 resCb[x][y]+2 CSign resCr[x][y]) / 5.

[0369] Alternatively, in mode 3 (with reconstructed Cr=C, Cb=(CSign) In the case of a single residual of C) / 2, the joint residual can be determined according to the following formula.

[0370] [Equation 23]

[0371] resJointC[x][y]=(4 resCr[x][y]+2 CSign resCb[x][y]) / 5.

[0372] [Table 2]

[0373] The above represents the reconstruction of the chroma residual. CSign refers to the sign value +1 or -1 specified in the slice header. resJointC[][] refers to the transmitted residual. In this table, the mode refers to TuCResMode (described later). The three joint chroma encoding modes in the table are supported only for I slices. For P and B slices, only mode 2 is supported. Therefore, for P and B slices, the syntax element tu_joint_cbcr_residual_flag can only be provided if both chroma cbf values ​​(e.g., tu_cbf_cb and tu_cbf_cr) are 1. Furthermore, the transform depth can be removed in the context modeling of tu_cbf_luma and tu_cbf_cb.

[0374] Implementation Method 1: QP Update Method Using ACT Qp_offset

[0375] As described above, QP can be updated to apply ACT. However, this QP update has several problems. For example, when using the above method, it is impossible to set different ACT Qp offsets for each color component. Furthermore, the derived qP value can be negative. Therefore, in the following embodiment, a method for applying clipping to Qp values ​​derived from ACT QP offset values ​​based on color component values ​​is described.

[0376] In the implementation, the quantization parameter qP can be derived as follows.

[0377] First, when cIdx has a value of 0, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0378] [Equation 24]

[0379] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0380] [Equation 25]

[0381] Alternatively, when cIdx has a value of 1, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0382] [Equation 26]

[0383] The quantization parameter qP can be updated as follows.

[0384] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP can be derived as shown in the following formula.

[0385] [Equation 27]

[0386] qP=Max(0, qP-(cu_act_enabled_flag[xTbY][yTbY] ? ActQpOffset:0))

[0387] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP can be derived as shown in the following equation.

[0388] [Equation 28]

[0389] In another implementation, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, the value of QpPrimeTsMin can be used instead of 0 to clip qP as shown in the following formula.

[0390] [Equation 29]

[0391] qP=Max(QpPrimeTsMin, qP-(cu_act_enabled_flag[xTbY][yTbY] ?ActQpOffset:0)

[0392] In another embodiment, the quantization parameter qP can be derived as follows.

[0393] First, when cIdx has a value of 0, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0394] [Formula 30]

[0395] qP=Qp' Y

[0396] ActQpOffset=5

[0397] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0398] [Equation 31]

[0399] qP=Qp' CbCr

[0400] ActQpOffset=5

[0401] Alternatively, when cIdx has a value of 1, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0402] [Equation 32]

[0403] qP=Qp' Cb

[0404] ActQpOffset=5

[0405] Alternatively, when cIdx has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0406] [Equation 33]

[0407] qP=Qp' Cr

[0408] ActQpOffset=3

[0409] The quantization parameter qP can be updated as follows.

[0410] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP can be derived as shown in the following formula.

[0411] [Formula 34]

[0412] qP=Max(0, qP-(cu_act_enabled_flag[xTbY][yTbY] ? ActQpOffset:0))

[0413] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP can be derived as shown in the following equation.

[0414] [Formula 35]

[0415] qP=Max(0, Max(QpPrimeTsMin, qP)-(cu_act_enabled_flag[xTbY][yTbY] ?ActQpOffset:0))

[0416] In another implementation, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, the value of QpPrimeTsMin can be used instead of 0 to clip qP as shown in the following formula.

[0417] [Formula 36]

[0418] qP=Max(QpPrimeTsMin, qP-(cu_act_enabled_flag[xTbY][yTbY] ?ActQpOffset:0))

[0419] In another embodiment, the quantization parameter qP can be derived as follows.

[0420] First, when cIdx has a value of 0, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0421] [Formula 37]

[0422] qP=Qp' Y

[0423] ActQpOffset=-5

[0424] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0425] [Formula 38]

[0426] qP=Qp' CbCr

[0427] ActQpOffset=-5

[0428] Alternatively, when cIdx has a value of 1, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0429] [Formula 39]

[0430] qP=Qp' Cb

[0431] ActQpOffset=-5

[0432] Alternatively, when cIdx has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0433] [Formula 40]

[0434] qP=Qp' Cr

[0435] ActQpOffset=-3

[0436] The quantization parameter qP can be updated as follows.

[0437] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP can be derived as shown in the following formula.

[0438] [Formula 41]

[0439] qP=Max(0, qP+(cu_act_enabled_flag[xTbY][yTbY] ? ActQpOffset:0))

[0440] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP can be derived as shown in the following equation.

[0441] [Equation 42]

[0442] qP=Max(0, Max(QpPrimeTsMin, qP)+(cu_act_enabled_flag[xTbY][yTbY] ?ActQpOffset:0))

[0443] In another implementation, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, the value of QpPrimeTsMin can be used instead of 0 to clip qP, as shown in the following formula.

[0444] [Formula 43]

[0445] qP=Max(QpPrimeTsMin, qP+(cu_act_enabled_flag[xTbY][yTbY] ?ActQpOffset:0))

[0446] In the above description, Y, Cb, and Cr can represent three color components. For example, in the ACT transform, Y can correspond to C0. Cb can correspond to C1 or Cg. Additionally, Cr can correspond to C2 or Co.

[0447] Additionally, the ACTQpOffset values ​​of -5, -5, and -3 for the three color components can be replaced with other values ​​or other variables.

[0448] Implementation Method 2: Signaling for QP Offset Adjustment of ACT

[0449] In the above embodiments, the ACT QP offset adjustment is fixed at -5, -5, and -3 for the Y, Cg, and Co components. In this embodiment, to provide greater flexibility in adjusting the ACT QP offset, a method for signaling the ACT QP offset will be described. The ACT QP offset can be signaled in the PPS as a parameter.

[0450] In the implementation method, it can be based on Figure 36 The syntax table is communicated to qp_offset via signals. Its syntax elements are as follows.

[0451] The syntax element pps_act_qp_offsets_present_flag can indicate whether there are syntax elements in PPS related to ACT QP offsets. For example, pps_act_qp_offsets_present_flag can indicate whether the syntax elements pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset (described later) are signaled by PPS.

[0452] For example, the first value of pps_act_qp_offsets_present_flag (e.g., 0) can indicate that pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset are not signaled via the PPS syntax table.

[0453] The second value of pps_act_qp_offsets_present_flag (e.g., 1) can instruct pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset to be signaled via the PPS syntax table.

[0454] When pps_act_qp_offsets_present_flag is not provided from the bitstream, pps_act_qp_offsets_present_flag can be deduced to have a first value (e.g., 0). For example, when a flag indicating whether an ACT is applied (e.g., sps_act_enabled_flag signaled in SPS) has a first value indicating that an ACT is not applied (e.g., 0), pps_act_qp_offsets_present_flag can be forced to have a first value (e.g., 0).

[0455] When the value of the syntax element `cu_act_enabled_flag` is the second value indicating that ACT is applied to the current encoding base (e.g., 1), the syntax elements `pps_act_y_qp_offset_plus5`, `pps_act_cb_qp_offset_plus5s`, and `pps_act_cr_qp_offset_plus3` can be used to determine the offsets of the quantization parameter values ​​`qP` applied to the luminance, Cb, and Cr components, respectively. When the values ​​of `pps_act_y_qp_offset_plus5`, `pps_act_cb_qp_offset_plus5`, and `pps_act_cr_qp_offset_plus3` are not present in the bitstream, each value can be set to 0.

[0456] Based on the syntax elements, the value of the variable PpsActQpOffsetY can be determined as pps_act_y_qp_offset_plus5-5. The value of the variable PpsActQpOffsetCb can be determined as pps_act_cb_qp_offset_plus5-5. Additionally, the value of the variable PpsActQpOffsetCr can be determined as pps_act_cb_qp_offset_plus3-3.

[0457] In this paper, ACT is not an orthogonal transformation, therefore 5, 5, and 3 can be used as constant offset values ​​to be subtracted. In the implementation, for bitstream matching, the values ​​of PpsActQpOffsetY, PpsActQpOffsetCb, and PpsActQpOffsetCr can have values ​​in the range of -12 to 12. Furthermore, according to the implementation, the Qp offset values, other than 5, 5, and 3, can be replaced with other constant values.

[0458] In another implementation, a more flexible ACT_QP offset can be used to adjust the QP. In the following implementation, an example of signaling the ACT QP offset in the bitstream is described. Therefore, the ACT QP offset can have a wider offset range. Consequently, the QP updated using the ACT QP offset is more likely to exceed the available range, thus necessitating pruning of the upper and lower limits of the updated QP (more detailed implementations will be described later in Implementations 6 and 7).

[0459] The variables PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, and PpsActQpOffsetCbCr, which indicate the ACT QP offset, can be values ​​derived using the ACT QP offset signaled via the bitstream or preset constants. For bitstream consistency, PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, and PpsActQpOffsetCbCr can have values ​​in the range of -12 to +12.

[0460] When a fixed value is not used, the value of the QP offset is signaled and its value is in the range of -12 to 12. In addition to pruning the lower limit of the derived QP value to avoid QP with negative values, it is necessary to prune the upper limit of the derived QP value.

[0461] To prevent qP from having negative values, the minimum value of qP can be forced to 0. Alternatively, the minimum value of qP can be set to a value determined by a syntax element signaled by a signal. For example, to signal the minimum value of qP when applying a transform skip mode, a syntax element QpPrimeTsMin indicating the value of qP applied when applying a transform skip mode can be used. The maximum value of qP can be limited to the maximum available value (e.g., 63) determined by the syntax element signaled by the signal.

[0462] In the implementation described above, the quantization parameter qP can be derived as follows. First, when cIdx has a value of 0, qP and the ACT Qp offset can be derived as shown in the following equation.

[0463] [Formula 44]

[0464] qP=Qp' Y

[0465] ActQpOffset=PpsActQpOffsetY

[0466] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0467] [Formula 45]

[0468] qP=Qp' CbCr

[0469] ActQpOffset=PpsActQpOffsetCbCr

[0470] Alternatively, when cIdx has a value of 1, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0471] [Formula 46]

[0472] qP=Qp' Cb

[0473] ActQpOffset=PpsActQpOffsetCb

[0474] Alternatively, when cIdx has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0475] [Formula 47]

[0476] qP=Qp' Cr

[0477] ActQpOffset=PpsActQpOffsetCr

[0478] In the implementation, the quantization parameter qP can be updated as follows. When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP can be derived as shown in the following equation.

[0479] [Formula 48]

[0480] qP=Clip3(0, 63, qP-(cu_act_enabled_flag[xTbY][yTbY] ? ActQpOffset:0))

[0481] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP can be derived as shown in the following equation.

[0482] [Formula 49]

[0483] qP=Clip3(0, 63, Max(QpPrimeTsMin, qP)-(cu_act_enabled_flag[xTbY][yTbY] ? ActQpOffset:0)

[0484] In another implementation, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, the minimum value of qP can be clipped using the value of QpPrimeTsMin instead of 0, as shown in the following formula.

[0485] [Formula 50]

[0486] The quantization parameter qP can be updated as follows.

[0487] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP can be derived as shown in the following formula.

[0488] qP=Clip3(0, 63, qP-(cu_act_enabled_flag[xTbY][yTbY] ? ActQpOffset:0))

[0489] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP can be derived as shown in the following equation.

[0490] qP=Clip3(QpPrimeTsMin, 63, qP -cu_act_enabled_flag[xTbY][yTbY] ?ActQpOffset:0)

[0491] In another implementation, the quantization parameter qP can be updated as follows.

[0492] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP can be derived as shown in the following formula.

[0493] [Equation 51]

[0494] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY] ?ActQpOffset:0))

[0495] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP can be derived as shown in the following equation.

[0496] [Equation 52]

[0497] qP=Clip3(0, 63+QpBdOffset, Max(QpPrimeTsMin, qP)+(cu_act_enabled_flag[xTbY][yTbY] ? ActQpOffset:0)

[0498] In another implementation, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, the minimum value of qP can be clipped using the value of QpPrimeTsMin instead of 0, as shown in the following formula.

[0499] [Formula 53]

[0500] The quantization parameter qP can be updated as follows.

[0501] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP can be derived as shown in the following formula.

[0502] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY] ?ActQpOffset:0))

[0503] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP can be derived as shown in the following equation.

[0504] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY] ? ActQpOffset:0)

[0505] Implementation Method 3: A method that allows ACT when performing chroma BDPCM

[0506] In this implementation, when BDPCM is applied to the luma component block, ACT can be applied to encode / decode the block. However, when BDPCM is applied to the chroma component block, ACT is restricted from being applied to encode / decode the block.

[0507] Furthermore, even when BDPCM is applied to chroma component blocks, ACT is also applied to the blocks, thereby improving coding efficiency. Figure 37 This illustrates an implementation of a syntax configuration that applies ACT even when BDPCM is applied to chroma component blocks. For example... Figure 37 As shown, by removing the condition that the value of cu_act_enabled_flag, which indicates whether ACT is applied to the current encoding base, is used to obtain the BDCPM syntax elements of the chroma components, the BDCPM syntax elements can be obtained regardless of whether ACT is applied to the chroma component blocks, and BDCPM encoding can be performed accordingly.

[0508] Implementation Method 4: Applying the ACT method even when performing encoding / decoding using CCLM

[0509] Both CCLM and ACT are designed to remove unwanted overlap between components. There is some overlap between CCLM and ACT, but even after applying all of them, the overlap between components cannot be completely removed. Therefore, by applying CCLM and ACT together, more overlap between components can be removed.

[0510] The following implementation describes an approach where CCLM and ACT are used together. During decoding, the decoding device may first apply CCLM, then apply ACT. When ACT is applied to both BDPCM and CCLM for the chroma components, the syntax table indicating this via signals can be as follows: Figure 38 The modifications are shown. Therefore, as... Figure 38 As shown in the syntax table, among the restrictions on signal notification of syntax elements related to intra_bdpcm_chroma and cclm, the if(!cu_act_enabled_flag) for signal notification of syntax elements based on whether ACT is not applied can be removed from the syntax table.

[0511] Implementation Method 5: Applying a flexible ACT Qp method including combined CbCr

[0512] When applying ACT mode, the prediction residual can be transformed from a color space (e.g., GBR or YcbCr) to the YCgCo color space. Additionally, the residual based on the transformation can be encoded in the YCgCo color space. As an implementation of the ACT core transform (transform kernel) for color space transformation, the following transform kernel can be used as described above.

[0513] [Formula 54]

[0514] [Formula 55]

[0515] As described in the above formula, the transformations of C0', C1', and C2' (in this paper, C0'=Y, C1'=Cg, C2'=Co) are not normalized. For example, the L2 norm does not have a value of 1. For example, the L2 norm of the transformation of each component can have a value of about 0.6 for C0' and C1', and a value of about 0.7 for C2'. In this paper, the L2 norm is obtained as the square root of the sum of the squares of the individual coefficients. For example, C0'=2 / 4 can be calculated. C0+1 / 4 C1+1 / 4 C2. Therefore, the norm of C0' can be calculated as (2 / 4) 2 / 4 + 1 / 4 1 / 4 + 1 / 4 The square root of 1 / 4. Therefore, this can be calculated as the square root of 6 / 16, and can be calculated as having a value of approximately 0.6.

[0516] When normalization transformation is not applied, the dynamic range of each component is irregular. Furthermore, this leads to a decrease in the coding performance of general video compression systems.

[0517] To compensate for the dynamic range of the residual signal, QP offset values ​​are sent to compensate for variations in the dynamic range of each transform component, enabling QP regulation. For example, this implementation can be applied to general QP regulation control methods for ACT transforms and combined with CbCr.

[0518] The individual color components are not encoded independently, but together, so that the method described above in Implementation 3 of the combined CbCr may cause variations in the dynamic range between the individual color components.

[0519] In the encoding and decoding method according to the embodiment, the ACT QP offset adjustment can be fixed at -5, which can also be applied to Y, Cg and Co.

[0520] In implementations, to provide flexible Qp control for each component and jointCbCr, different ACT Qp offsets can be used for Y, Cb, Cr, and / or jointCbCr. The ACT Qp offset values ​​can be determined based on the component index and / or jointCbCr and / or jointCbCr mode.

[0521] To indicate the ACT Qp offset, ppsActQpOffsetY, ppsActQpOffsetCb, and ppsActQpOffsetCr can be used. Additionally, ppsActQpOffsetCbCr can be used for the ACT QP offset in joint CbCr mode 2, where all Cb and Cr components have non-zero values ​​in a CBF. These values ​​(e.g., ppsActQpOffsetY, ppsActQpOffsetCb, ppsActQpOffsetCr, and ppsActQpOffsetCbCr) can be predetermined or signaled via a bitstream. The ACT QP offset in joint CbCr mode can be set using another method or set to another value.

[0522] In implementation, ACT Qp offsets -5, -5, and -3 can be used for Y, Cb, and Cr, and ACT Qp offset -4 can be used for combined CbCr.

[0523] In another implementation, ACT Qp offsets -5, -4, and -3 can be used for Y, Cb, and Cr, and ACT Qp offset -3 can be used for the joint CbCr mode, where the value of tu_cbf_cb is not 0.

[0524] In another implementation, the ACT QP offset of the joint CbCr mode 2 can have its own offset value. For another joint CbCr mode, the ACT QP offset can use the offset of the corresponding component. For example, the quantization parameter qP can be determined as follows. First, when cIdx has a value of 0, qP and the ACT QP offset can be derived as shown in the following equation.

[0525] [Formula 56]

[0526] qP=Qp' Y

[0527] ActQpOffset=ppsActQpOffsetY

[0528] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0529] [Formula 57]

[0530] qP=Qp' CbCr

[0531] ActQpOffset=ppsActQpOffsetCbCr

[0532] Alternatively, when cIdx has a value of 1, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0533] [Formula 58]

[0534] qP=Qp' Cb

[0535] ActQpOffset=ppsActQpOffsetCb

[0536] Alternatively, when cIdx has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0537] [Formula 59]

[0538] qP=Qp' Cr

[0539] ActQpOffset=ppsActQpOffsetCr

[0540] In this implementation, the quantization parameter qP can be updated as follows.

[0541] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP can be derived as shown in the following formula.

[0542] [Formula 60]

[0543] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY] ?ActQpOffset:0))

[0544] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP can be derived as shown in the following equation.

[0545] [Formula 61]

[0546] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY] ? ActQpOffset:0)

[0547] In another implementation, for the joint CbCr mode where tu_cbf_cb != 0 (e.g., in modes 1 and 2), ppsActQpOffsetCb can be used to determine the offset of the joint CbCr. Alternatively, for the joint CbCr mode where tu_cbf_cb == 0 (e.g., in mode 3), ppsActQpOffsetCr can be used to determine the offset of the joint CbCr. For example, the above implementation can be modified and applied as follows.

[0548] The quantization parameter qP can be updated as follows. First, when cIdx has a value of 0, qP and the ACT Qp offset can be derived as shown in the following equation.

[0549] [Formula 62]

[0550] qP=Qp' Y

[0551] ActQpOffset=ppsActQpOffsetY

[0552] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP can be derived as shown in the following equation.

[0553] [Formula 63]

[0554] qP=Qp' CbCr

[0555] Alternatively, when cIdx has a value of 1, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0556] [Formula 64]

[0557] qP=Qp' Cb

[0558] ActQpOffset=ppsActQpOffsetCb

[0559] Alternatively, when cIdx has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0560] [Formula 65]

[0561] qP=Qp' Cr

[0562] ActQpOffset=ppsActQpOffsetCr

[0563] When cIdx does not have a value of 0 and TuCResMode[xTbY][yTbY] does not have a value of 0, the ACT Qp offset of the joint CbCr mode can be determined according to the following pseudocode.

[0564] [Formula 66]

[0565] if (TuCResMode[xTbY][yTbY] is not equal to 1 or 2)

[0566] ActQpOffset=ppsActQpOffsetCb;

[0567] else

[0568] ActQpOffset=ppsActQpOffsetCr;

[0569] In the implementation, the quantization parameter qP can be updated as follows. When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP can be derived as shown in the following equation.

[0570] [Formula 67]

[0571] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY] ?ActQpOffset:0))

[0572] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP can be derived as shown in the following equation.

[0573] [Formula 68]

[0574] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY] ? ActQpOffset:0)

[0575] In another implementation, regardless of the joint CbCr mode, ppsActQpOffsetY is used when the component index is Y, ppsActQpOffsetCb is used when the component index is Cb, and ppsActQpOffsetCr is used when the component index is Cr, thereby deriving qP. For example, the quantization parameter qP can be derived as follows.

[0576] First, when cIdx has a value of 0, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0577] [Formula 69]

[0578] qP=Qp' Y

[0579] ActQpOffset=ppsActQpOffsetY

[0580] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0581] [Formula 70]

[0582] qP=Qp' CbCr

[0583] ActQpOffset=(cIdx==1)? ppsActQpOffsetCb : ppsActQpOffsetCr

[0584] Alternatively, when cIdx has a value of 1, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0585] [Formula 71]

[0586] qP=Qp' Cb

[0587] ActQpOffset=ppsActQpOffsetCb

[0588] Alternatively, when cIdx has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0589] [Equation 72]

[0590] qP=Qp' Cr

[0591] ActQpOffset=ppsActQpOffsetCr

[0592] The quantization parameter qP can be updated as follows.

[0593] When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, qP can be derived as shown in the following formula.

[0594] [Formula 73]

[0595] qP=Clip3(0, 63+QpBdOffset, qP+(cu_act_enabled_flag[xTbY][yTbY] ?ActQpOffset:0))

[0596] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1, qP can be derived as shown in the following equation.

[0597] [Formula 74]

[0598] qP=Clip3(QpPrimeTsMin, 63+QpBdOffset, qP+cu_act_enabled_flag[xTbY][yTbY] ? ActQpOffset:0)

[0599] Implementation Method 6: A method for signaling the ACT Qp offset including combined CbCr

[0600] The following describes an example of using the bitstream to signal the ACT QP offset for greater flexibility. The ACT QP offset can be signaled via SPS, PPS, frame header, slice header, or other types of header sets. The combined CbCr ACT QP offset can be signaled separately, or it can be derived from the ACT QP offsets of Y, Cb, and Cr.

[0601] Without loss of generality, Figure 39 This shows an example of a syntax table in PPS that uses signals to notify the ACT Qp offset. For example... Figure 39 In this implementation, an ACT Qp offset can be signaled for the joint CbCr. This will be described... Figure 39The syntax elements indicated in the syntax table.

[0602] The syntax element pps_act_qp_offsets_present_flag can indicate whether a syntax element related to the ACT QP offset exists in the PPS. For example, pps_act_qp_offsets_present_flag can indicate whether to signal the syntax elements pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4 (which will be described later) as part of the PPS.

[0603] For example, the first value of pps_act_qp_offsets_present_flag (e.g., 0) can indicate that pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4 are not notified by signaling through the PPS syntax table.

[0604] The second value of pps_act_qp_offsets_present_flag (e.g., 1) can instruct pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, and pps_act_cbcr_qp_offset_plusX4 to be signaled via the PPS syntax table.

[0605] When pps_act_qp_offsets_present_flag is not provided from the bitstream, pps_act_qp_offsets_present_flag can be deduced to have a first value (e.g., 0). For example, when a flag indicating whether an ACT is applied (e.g., sps_act_enabled_flag signaled in SPS) has a first value indicating that an ACT is not applied (e.g., 0), pps_act_qp_offsets_present_flag can be forced to have a first value (e.g., 0).

[0606] When the value of the syntax element `cu_act_enabled_flag` is the second value (e.g., 1) indicating that ACT is applied to the current encoding base, the syntax elements `pps_act_y_qp_offset_plusX1`, `pps_act_cb_qp_offset_plusX2`, `pps_act_cr_qp_offset_plusX3`, and `pps_act_cbcr_qp_offset_plusX4` can be used to determine the offsets of the quantization parameter values ​​`qP` applied to the luminance, Cb, Cr components, and joint CbCr components, respectively. When `pps_act_y_qp_offset_plusX1`, `pps_act_cb_qp_offset_plusX2`, `pps_act_cr_qp_offset_plusX3`, and `pps_act_cbcr_qp_offset_plusX4` are not present in the bitstream, each value can be set to 0.

[0607] Based on the syntax elements, the values ​​of variables PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, and PpsActQpOffsetCbCr can be determined as shown in the following formula.

[0608] [Formula 75]

[0609] PpsActQpOffsetY=pps_act_y_qp_offset_plusX1-X1

[0610] PpsActQpOffsetCb=pps_act_cb_qp_offset_plusX2 -X2

[0611] PpsActQpOffsetCr=pps_act_cr_qp_offset_plusX3-X3

[0612] PpsActQpOffsetCbCr=pps_act_cbcr_qp_offset_plusX4-X4

[0613] In this document, X1, X2, X3, and X4 may indicate predetermined constant values. These may be the same value, different values, or only some of them may have the same value.

[0614] In the implementation, for bitstream matching, the values ​​of PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, and PpsActQpOffsetCbCr can be restricted to values ​​in the range of -12 to 12.

[0615] Based on the determination of the variables, the quantization parameter qP can be determined as follows. First, when cIdx has a value of 0, qP and the ACT Qp offset can be derived as shown in the following equation.

[0616] [Formula 76]

[0617] qP=Qp' Y

[0618] ActQpOffset=PpsActQpOffsetY

[0619] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0620] [Formula 77]

[0621] qP=Qp' CbCr

[0622] ActQpOffset=PpsActQpOffsetCbCr

[0623] Alternatively, when cIdx has a value of 1, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0624] [Formula 78]

[0625] qP=Qp' Cb

[0626] ActQpOffset=PpsActQpOffsetCb

[0627] Alternatively, when cIdx has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0628] [Formula 79]

[0629] qP=Qp' Cr

[0630] ActQpOffset=PpsActQpOffsetCr

[0631] In another implementation of signaling ACT Qp offsets, multiple ACT QP offsets can be signaled for different joint CbCr modes identified as mode A and mode B.

[0632] Joint CbCr mode A can refer to a jointCbCr mode where tu_cbf_cb has a non-zero value, such as modes 1 and 2 in Table 2 above. Joint CbCr mode B can refer to a jointCbCr mode where tu_cbf_cb has a value of 0, such as mode 3 in Table 2 above. The corresponding changes in syntax are based on... Figure 40 The description will be in the middle. Figure 40 The syntax elements indicated in the syntax table.

[0633] When the value of the syntax element cu_act_enabled_flag is a second value (e.g., 1) indicating that ACT is applied to the current encoding base, the syntax elements pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, pps_act_cbcr_qp_offset_modeA_plusX4, and pps_act_cbcr_qp_offset_modeB_plusX5 can be used to determine the offset of the quantization parameter value qP applied to the luminance, Cb, Cr components, and joint CbCr components, respectively. When the values ​​of pps_act_y_qp_offset_plusX1, pps_act_cb_qp_offset_plusX2, pps_act_cr_qp_offset_plusX3, pps_act_cbcr_qp_offset_modeA_plusX4, and pps_act_cbcr_qp_offset_modeB_plusX5 do not exist in the bitstream, each value can be set to 0.

[0634] Based on the syntax elements, the values ​​of variables PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, PpsActQpOffsetCbCrModeA, and PpsActQpOffsetCbCrModeB can be determined as shown in the following formula.

[0635] [Formula 80]

[0636] PpsActQpOffsetY=pps_act_y_qp_offset_plusX1-X1

[0637] PpsActQpOffsetCb=pps_act_cb_qp_offset_plusX2 -X2

[0638] PpsActQpOffsetCr=pps_act_cr_qp_offset_plusX3-X3

[0639] PpsActQpOffsetCbCrModeA=pps_act_cbcr_qp_offset_modeA_plusX4-X4

[0640] PpsActQpOffsetCbCrModeB=pps_act_cbcr_qp_offset_modeB_plusX5-X5

[0641] In this document, X1, X2, X3, X4, and X5 can indicate predetermined constant values. These can be the same value, different values, or only some can have the same value. In the implementation, for bitstream matching, the values ​​of PpsActQpOffsetY, PpsActQpOffsetCb, PpsActQpOffsetCr, PpsActQpOffsetCbCrModeA, and PpsActQpOffsetCbCrModeB can be restricted to values ​​in the range of -12 to 12.

[0642] Based on the determination of the variables, the quantization parameter qP can be determined as follows. First, when cIdx has a value of 0, qP and the ACT Qp offset can be derived as shown in the following equation.

[0643] [Formula 81]

[0644] qP=Qp' Y

[0645] ActQpOffset=PpsActQpOffsetY

[0646] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP can be derived as shown in the following equation.

[0647] [Equation 82]

[0648] qP=Qp' CbCr

[0649] Alternatively, when cIdx has a value of 1, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0650] [Formula 83]

[0651] qP=Qp' Cb

[0652] ActQpOffset=PpsActQpOffsetCb

[0653] Alternatively, when cIdx has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0654] [Formula 84]

[0655] qP=Qp' Cr

[0656] ActQpOffset=PpsActQpOffsetCr

[0657] Additionally, when cIdx does not have a value of 0 and TuCResMode[xTbY][yTbY] does not have a value of 0, the ACT Qp offset can be derived as shown in the following equation.

[0658] [Formula 85]

[0659] Furthermore, in another implementation, when TuCResMode[xTbY][yTbY] has a value of 2, ActQpOffset can be derived as shown in the following formula.

[0660] [Formula 86]

[0661] In another implementation that uses signals to notify ACT Qp of the offset, it can be as follows: Figure 41 The syntax table only uses signals to indicate the ACT QP offsets of Y, Cb, and Cr. The ACT QP offset of the joint CbCr can be derived from PpsActQpOffsetY, PpsActQpOffsetCb, and / or PpsActQpOffsetCr.

[0662] In one implementation, the ACT Qp offset of CbCr can be set to the value of PpsActQpOffsetCb. In another implementation, the ACT Qp offset of CbCr can be set to the same value as PpsActQpOffsetCb in the case of a joint CbCr mode where tu_cbf_cb has a non-zero value, or it can be set to the same value as PpsActQpOffsetCr in the case of a joint CbCr mode where tu_cbf_cb has a value of 0. Alternatively, it can be set in the opposite way.

[0663] Figure 41 This is a view illustrating another implementation of the syntax table in PPS that uses signals to notify the ACT Qp offset. According to Figure 41 The grammar elements are determined, and the quantization parameter qP can be determined as follows. First, when cIdx has a value of 0, qP and the ACT Qp offset can be derived as shown in the following equation.

[0664] [Formula 87]

[0665] qP=Qp' Y

[0666] ActQpOffset=PpsActQpOffsetY

[0667] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0668] [Formula 88]

[0669] qP=Qp' CbCr

[0670] ActQpOffset=(cIdx==1)? PpsActQpOffsetCb : PpsActQpOffsetCr

[0671] In another embodiment, the value of ActQpOffset can be determined as follows.

[0672] [Formula 89]

[0673] ActQpOffset=(tu_cbf_cb[xTbY][yTbY])? PpsActQpOffsetCb :PpsActQpOffsetCr

[0674] Alternatively, when cIdx has a value of 1, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0675] [Formula 90]

[0676] qP=Qp' Cb

[0677] ActQpOffset=PpsActQpOffsetCb

[0678] Alternatively, when cIdx has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0679] [Formula 91]

[0680] qP=Qp' Cr

[0681] ActQpOffset=PpsActQpOffsetCr

[0682] Implementation Method 7: Signaling ACT Qp offset at multiple levels

[0683] In implementations, the ACT QP offset can be signaled at multiple levels. In addition to signaling the ACT QP offset at one level (e.g., PPS) as in Implementation 6 above, the ACT QP offset can be signaled at lower levels (e.g., slice head, frame head, or other types of heads suitable for Qp control).

[0684] The following will describe two implementation methods. Figure 42 and Figure 43 This illustrates an example of notifying the ACT QP offset via signals through the slice header and the frame header. This allows for signaling the ACT QP offset at multiple levels.

[0685] The following will describe Figure 42 and Figure 43 The syntax elements shown are: pps_slice_act_qp_offsets_present_flag, which indicates whether the syntax elements slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset (described later) exist in the slice header.

[0686] For example, the first value of pps_slice_act_qp_offsets_present_flag (e.g., 0) can indicate that slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset do not exist in the slice header.

[0687] For example, the second value of pps_slice_act_qp_offsets_present_flag (e.g., 1) can indicate the presence of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset in the slice header.

[0688] The syntax elements slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset can indicate the offset of the quantization parameter value qP for the luminance, Cb, Cr components, and the joint CbCr component, respectively. The values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset can be restricted to values ​​in the range of -12 to 12. When slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset are not present in the bitstream, each value can be set to 0. The values ​​of PpsActQpOffsetY+slice_act_y_qp_offset, PpsActQpOffsetCb+slice_act_cb_qp_offset, PpsActQpOffsetCr+slice_act_cr_qp_offset, and PpsActQpOffsetCbCr+slice_act_cbcr_qp_offset can be restricted to values ​​also in the range of -12 to 12.

[0689] Various modified implementations can be applied to signal the ACT QP offset of joint CbCr at the PPS level. For example, a single QP offset can be signaled for joint CbCr, multiple ACT QP offsets can be signaled for joint CbCr in different modes, or the ACT QP offset of joint CbCr can be signaled without signaling. When signaling via the slice header, this can be derived using ACTQpOffsets of modes Y, Cb, and Cr and / or jointCbCr.

[0690] Figure 44 and Figure 45 Two modified implementations are shown in the figure. Figure 44 This illustrates an implementation where the ACT Qp offset is signaled in the slice header. Figure 45 This illustrates another implementation where the ACT Qp offset is signaled in the slice header. Figure 45In this implementation, the ACT Qp offsets for Y, Cb, and Cr can be communicated using only signals, and the slice-level ACT Qp offset for joint CbCr can be derived from slice_act_y_qp_offset, slice_act_cb_qp_offset, and / or slice_act_cr_qp_offset. This can be determined based on the mode type of jointCbCr. In one implementation, the slice-level ACT Qp offset for CbCr can be set to the same value as slice_act_cb_qp_offset. In another implementation, in the case of a joint CbCr mode where tu_cbf_cb has a non-zero value, the slice-level ACT Qp offset for joint CbCr can be set to the same value as slice_act_cb_qp_offset. Additionally, in the case of a jointCbCr mode where tu_cbf_cb has a value of 0, the slice-level ACT Qp offset for joint CbCr can be set to the same value as slice_act_cr_qp_offset.

[0691] In another implementation, the syntax elements can be notified by signals in the slice header or frame header. To achieve this, encoding / decoding can be performed as follows.

[0692] - The flag pps_picture_slice_act_qp_offsets_present_flag, which indicates whether there is an ACT Qp offset in the picture header or slice header, can be notified by a signal in PPS.

[0693] - When ACT is applicable and the value of pps_picture_slice_act_qp_offsets_present_flag is the second value (e.g., 1), the flag pic_act_qp_offsets_present_flag indicating the presence of ACT Qp offsets in the picture header is signaled in the picture header. In this document, the second value of pic_act_qp_offsets_present_flag (e.g., 1) can indicate the ACT Qp offsets of all slices of the picture corresponding to the picture header provided in the picture header.

[0694] - A first value (e.g., 0) for pic_act_qp_offsets_present_flag can indicate the ACT Qp offsets for all slices of the picture that do not have a corresponding picture header in the picture header. For example, when ACT applies and pps_picture_slice_act_qp_offsets_present_flag is a second value (e.g., 1) and pic_act_qp_offsets_present_flag is a first value (e.g., 0), the ACT Qp offsets for the slices can be provided in the slice header.

[0695] Figure 46 This is a view of the syntax table of the PPS that uses signals to notify pps_pic_slice_act_qp_offsets_present_flag. The syntax element pps_pic_slice_act_qp_offsets_present_flag can indicate whether the ACT Qp offset is provided in the frame header and / or slice header. For example, a first value (e.g., 0) of pps_pic_slice_act_qp_offsets_present_flag can indicate that the ACT Qp offset is not provided in the frame header and slice header. A second value (e.g., 1) of pps_pic_slice_act_qp_offsets_present_flag can indicate that the ACT Qp offset is provided in the frame header or slice header. When pps_pic_slice_act_qp_offsets_present_flag is not provided in the bitstream, the value of pps_pic_slice_act_qp_offsets_present_flag can be determined to be the first value (e.g., 0).

[0696] Figure 47 This is a view of the syntax table showing the frame header that signals the ACT Qp offset. The syntax element `pic_act_qp_offsets_present_flag` indicates whether the ACT Qp offset is provided in the frame header. A first value of `pic_act_qp_offsets_present_flag` (e.g., 0) indicates that the ACT Qp offset is provided in the slice header, not the frame header. A second value of `pic_act_qp_offsets_present_flag` (e.g., 1) indicates that the ACT Qp offset is provided in the frame header. When a value for `pic_act_qp_offsets_present_flag` is not provided in the bitstream, the value can be determined to be 0.

[0697] Figure 48This is a view showing the syntax table of the slice header that signals the ACT Qp offset. Figure 48 In the syntax table, the syntax elements slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset indicate the offset of the quantization parameter value qP for the luminance, Cb, and Cr components. The values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset can be in the range of -12 to 12. Additionally, the values ​​of PpsActQpOffsetY+slice_act_y_qp_offset, PpsActQpOffsetCb+slice_act_cb_qp_offset, and PpsActQpOffsetCr+slice_act_cr_qp_offset can also be restricted to the range of -12 to 12.

[0698] Furthermore, if the values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, slice_act_cr_qp_offset, and slice_act_cbcr_qp_offset are not provided in the bitstream, the values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, and slice_act_cr_qp_offset can be determined to be 0 when the value of pps_pic_slice_act_qp_offsets_present_flag is the first value (e.g., 0). Alternatively, when the value of pps_pic_slice_act_qp_offsets_present_flag is the second value (e.g., 1), the values ​​of slice_act_y_qp_offset, slice_act_cb_qp_offset, and slice_act_cr_qp_offset can be determined to be the same as pps_act_y_qp_offset, pps_act_cb_qp_offset, and pps_act_cr_qp_offset, respectively.

[0699] Furthermore, when there is an ACT Qp offset in both the slice header and the picture header, the final offset value used to derive the qP value can be determined by adding the offset value signaled in the PPS to the offset value signaled in the slice header or the picture header.

[0700] More specifically, in the implementation, the quantization parameter qP can be determined as follows. First, when cIdx has a value of 0, qP and the ACT Qp offset can be derived as shown in the following equation.

[0701] [Equation 92]

[0702] qP=Qp' Y

[0703] ActQpOffset=PPsQpOffsetY+slice_act_y_qp_offset

[0704] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0705] [Formula 93]

[0706] qP=Qp' CbCr

[0707] ActQpOffset=PPsQpOffsetCbCr+slice_act_CbCr_qp_offset

[0708] Alternatively, when cIdx has a value of 1, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0709] [Formula 94]

[0710] qP=Qp' Cb

[0711] ActQpOffset=PpsActQpOffsetCb+slice_act_Cb_qp_offset

[0712] Alternatively, when cIdx has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0713] [Formula 95]

[0714] qP=Qp' Cr

[0715] ActQpOffset=PpsActQpOffsetCr+slice_act_Cr_qp_offset

[0716] In another implementation, when multiple ACT Qp offsets of the combined CbCr are signaled, the ActQpOffset of the combined CbCr can be determined as follows.

[0717] First, when cIdx has a value of 0, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0718] [Formula 96]

[0719] qP=Qp' Y

[0720] ActQpOffset=PPsQpOffsetY+slice_act_y_qp_offset

[0721] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP can be derived as shown in the following equation.

[0722] [Formula 97]

[0723] qP=Qp' CbCr

[0724] Alternatively, when cIdx has a value of 1, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0725] [Formula 98]

[0726] qP=Qp' Cb

[0727] ActQpOffset=PpsActQpOffsetCb+slice_act_Cb_qp_offset

[0728] Alternatively, when cIdx has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0729] [Formula 99]

[0730] qP=Qp' Cr

[0731] ActQpOffset=PpsActQpOffsetCr+slice_act_Cr_qp_offset

[0732] Additionally, when cIdx does not have a value of 0 and TuCResMode[xTbY][yTbY] does not have a value of 0, the ACT Qp offset can be derived as shown in the following equation.

[0733] [Formula 100]

[0734] In another embodiment, when the ACT Qp offset for the combined CbCr is not provided, the qP and ActQpOffsets for the Y, Cb, and / or Cr components are determined, and the ActQpOffset for the combined CbCr can be determined using the ACT Qp offsets for the Y, Cb, and / or Cr components as follows. For example, in the above embodiment, when TuCResMode[xTbY][yTbY] related to Equation 97 has a value of 2, the calculation steps for qP can be modified and performed as follows.

[0735] "Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation."

[0736] [Formula 101]

[0737] In another embodiment, the value of ActQpOffset can be determined by the following formula.

[0738] [Equation 102]

[0739] ActQpOffset=(tu_cbf_cb[xTbY][yTbY]) ? (PPsQpOffsetCb+slice_act_Cb_qp_offset) : (PPsQpOffsetCr+slice_act_Cr_qp_offset)

[0740] Implementation Method 8: Method for Notifying Multiple ACT Qp Offset Sets Using Signals

[0741] In this embodiment, a method for using a list of ACT Qp offsets will be described. For this purpose, the following processing can be performed.

[0742] a) Multiple sets of ACT Qp offsets can be signaled in list form within a parameter set (e.g., SPS or PPS). Each set in the list can include ACT Qp offsets for the Y, Cb, Cr, and joint CbCr components. For simplicity, the list of ACT Qp offsets can be signaled within the same parameter set as the list of chromaticity Qp offsets.

[0743] b) The number of ACT Qp offset sets in the list can be the same as the number of chroma Qp offset sets notified by signals in the PPS.

[0744] c) As the ACT Qp offset used to derive the qP of each coding basis, the ACT Qp offset can be a list of indices belonging to the chroma Qp offsets with coding basis (e.g., cu_chroma_qp_offset_idx).

[0745] d) As an alternative implementation to b) and c), the following operations may be performed.

[0746] - The number of ACT Qp offset sets in the list can be signaled. The number of ACT Qp offset sets in the list may differ from the number of chroma Qp offset sets.

[0747] - When ACT is applicable, the index of the index indicating the ACT Qp offset used for encoding can be signaled.

[0748] Without departing from the above concepts, the syntax for signaling the list of ACT Qp offsets can be as follows: Figure 49 As shown in the diagram. For example, when cu_act_enabled_flag has a value of 1, pps_act_y_qp_offset, pps_act_cb_qp_offset, pps_act_cr_qp_offset, and pps_act_cbcr_qp_offset can be used to determine the offset of the quantization parameter value qP to be applied to the luminance, Cb, Cr components, and joint CbCr, respectively.

[0749] When the values ​​of pps_act_y_qp_offset, pps_act_cb_qp_offset, pps_act_cr_qp_offset, and pps_act_cbcr_qp_offset do not exist, each value can be deduced to be 0.

[0750] When the value of cu_act_enabled_flag is the second value (e.g., 1) and the value of cu_chroma_qp_offset_flag is the second value (e.g., 1), act_y_qp_offset_list[i], act_cb_qp_offset_list[i], act_cr_qp_offset_list[i], and act_cbcr_qp_offset_list[i] can be used to determine the offsets of the quantization parameter values ​​qP applied to the luminance, Cb, Cr components, and joint CbCr components, respectively. When there are no values ​​for act_y_qp_offset_list[i], act_cb_qp_offset_list[i], act_cr_qp_offset_list[i], and act_cbcr_qp_offset_list[i], each value can be deduced to be 0.

[0751] In this embodiment, the quantization parameter qP can be determined as follows. First, when cIdx has a value of 0, qP and the ACT Qp offset can be derived as shown in the following equation.

[0752] [Equation 103]

[0753] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0754] [Equation 104]

[0755] qP=Qp' CbCr

[0756] ActQpOffset=pps_act_cbcr_qp_offset+(cu_chroma_qp_offset_flag) ? act_cbcr_qp_ offset_list[cu_chroma_qp_offset_idx] : 0+slice_act_cbcr_qp_offset

[0757] Alternatively, when cIdx has a value of 1, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0758] [Equation 105]

[0759] qP=Qp' Cb

[0760] ActQpOffset=pps_act_cb_qp_offset+(cu_chroma_qp_offset_flag) ? act_cb_qp_offset _list[cu_chroma_qp_offset_idx] : 0+slice_act_cb_qp_offset

[0761] Alternatively, when cIdx has a value of 2, the offsets qP and ACT Qp can be derived as shown in the following equation.

[0762] [Equation 106]

[0763] qP=Qp' Cr

[0764] ActQpOffset=pps_act_cr_qp_offset+(cu_chroma_qp_offset_flag) ? act_cr_qp_offset_ list[cu_chroma_qp_offset_idx] : 0+slice_act_cr_qp_offset

[0765] Implementation Method 9: ACT color space conversion method applicable to both lossless and lossy encoding

[0766] The transformations between color spaces based on the matrices used for forward and backward transformations described above can be organized as follows.

[0767] [Table 3]

[0768] Because some values ​​are lost during the processing of Co and Cg, the conversion cannot reconstruct the original state. For example, when a sample value in the RGB color space is converted to a sample value in the YCgCo color space and then the resulting value is converted back to a sample value in the RGB color space, the original sample value is not fully reconstructed. Therefore, the conversion according to Table 3 cannot be used for lossless encoding. It is necessary to improve the color space conversion algorithm so that no loss of sample values ​​occurs after the color space conversion, even when lossless encoding is applied. Embodiments 9 and 10 describe color space conversion algorithms that can be applied to both lossless and lossy encoding.

[0769] In the following embodiments, a method for performing ACT is described using a color space conversion that is reversible to the original state suitable for both lossy and lossless encoding. This reversible color space conversion can be applied to the encoding and decoding methods described above. The ACT Qp offset can also be adjusted for the color space conversions described below. The color space conversion according to the embodiments can be performed as shown in the following formula. For example, a forward conversion from the GBR color space to the YCgCo color space can be performed according to the following formula.

[0770] [Equation 107]

[0771] Co = RB;

[0772] t = B + (Co >> 1);

[0773] Cg=Gt;

[0774] Y = t + (Cg >> 1);

[0775] Additionally, a backward conversion from the YCgCo color space to the GBR color space can be performed according to the following formula.

[0776] [Equation 108]

[0777] t = Y - (Cg >> 1)

[0778] G=Cg+t

[0779] B = t - (Co >> 1)

[0780] R = Co + B

[0781] The conversion between the YCgCo and RGB color spaces according to the above formulas is reversible to the original state. That is, color space conversions according to these formulas support perfect reconstruction. For example, sample values ​​are preserved even after a backward conversion is performed following a forward conversion. Therefore, color space conversions according to these formulas can be called reversible YCgCo-R color conversions. In this paper, R can stand for reversible, meaning that reconstruction to the original state is achieved. The YCgCo-R conversion can be provided by increasing the bit depth of Cg and Co by 1 compared to existing conversions. If this condition is met, other types of convertible conversions can be used similarly to the conversions described above.

[0782] Since the conversion shown in the above formula has a different norm value than the conversion described above, the ACT Qp offset of Y, Cg, and Co can be adjusted to compensate for the dynamic range changes caused by the color space conversion.

[0783] It has been described that when the above transformation is applied, the QCT Qp offset according to the embodiment can have values ​​of (-5, -5, -5) for Y, Cg, and Co. However, when the reversible transformation according to this embodiment is applied, a value other than (-5, -5, -5) can be specified as the QCT Qp offset according to the embodiment. For example, the values ​​of Y, Cg, and Co (-5, 1, 3) can be used as the QCT Qp offset according to the embodiment.

[0784] In another embodiment, the ACT QP offset can be notified via a bit stream signal, as in embodiment 6 or 7 described above.

[0785] For example, when the above YCgCo-R transformation is used with the ACT QP offset of (-5, 1, 3), as shown in the figure below, no coding loss is observed in lossy coding environments (e.g., QP 22, 27, 32, 37). Furthermore, it is observed that when ACT is applied, an additional 5% coding performance is achieved in achieving lossless coding.

[0786] [Table 4]

[0787] The VVC specification, including the integrated ACT matrix, can be described in the following table.

[0788] [Table 5]

[0789] For example, a residual sample array r of size (nTbW)×(nTbH) Y r Cb and r Cr The following updates are possible.

[0790] [Equation 109]

[0791] tmp=r Y [x][y]-(r Cb [x][y]>>1)

[0792] r Y [x][y]=tmp+r Cb [x][y]

[0793] r Cb [x][y]=tmp-(r Cr [x][y]>>1)

[0794] r Cr [x][y]=r Cb [x][y]+r Cr [x][y]

[0795] Implementation Method 10: ACT Execution Method for Performing Multiple Color Transitions Based on Explicit Signaling

[0796] In this embodiment, at least one color conversion can be performed via ACT. Which color conversion to perform can be determined by flags signaled in the bitstream. These flags can be signaled at multiple levels or at an identifiable granularity, such as SPS, PPS, frame header, and slice.

[0797] In implementation, a predetermined flag can be signaled to indicate which ACT (Activity) to apply. For example, when the flag has a value of 1, an ACT based on reversible color transformation can be applied. When the flag has a value of 0, an ACT based on irreversible color transformation can be applied.

[0798] In another implementation, a predetermined flag of the ACT can be signaled to indicate which color transition to use. Figure 50 The text shows an example of the syntax for signal notification in SPS. It will be described... Figure 50 The syntax element `sps_act_reversible_conversion` indicates whether an irreversible conversion to the original state is used. A first value of `sps_act_reversible_conversion` (e.g., 0) indicates that `ACT` uses an irreversible conversion to the original state. A second value of `sps_act_reversible_conversion` (e.g., 1) indicates that `ACT` uses a reversible conversion to the original state.

[0799] Therefore, the variable lossyCoding, which indicates whether lossy coding is performed, can be set as follows.

[0800] [Formula 110]

[0801] lossyCoding=(!sps_act_reversible_conversion)

[0802] By using the lossyCoding flag, the pseudocode for the backward conversion from YcgCo to GBR performed by the decoding device during the decoding process can be expressed as follows.

[0803] [Formula 111]

[0804] If (sps_act_reversible_conversion==1)

[0805] {

[0806] / / YCgCo-R reversible conversion

[0807] t = Y - (Cg >> 1)

[0808] G=Cg+t

[0809] B = t - (Co >> 1)

[0810] R = Co + B

[0811] }

[0812] else {

[0813] t=Y-Cg

[0814] G=Y+Cg

[0815] B=t-Co

[0816] R=t+Co

[0817] }

[0818] Therefore, the VVC specification shown in Table 5 of Implementation 9 can be modified as shown in the following table.

[0819] [Table 6]

[0820] According to the table above, the residual update process using color space conversion can use the following parameters as input for this process.

[0821] - Variable nTbW, indicating the block width

[0822] - Variable nTbH indicates the block height.

[0823] -A (nTbW)×(nTbH) size array r of luminance residual samples Y , by element r Y Composed of [x][y], -A (nTbW)×(nTbH) array of chromaticity residual samples Cb , by element r Cb Composed of [x][y], -A (nTbW)×(nTbH) array of chromaticity residual samples Cr , by element r Cr Composed of [x][y], The output of this process is as follows.

[0824] - Update array r with size (nTbW)×(nTbH) of luminance residual samples Y , - Update array r of size (nTbW)×(nTbH) for chromaticity residual samples Cb , -Update array r with (nTbW)×(nTbH)-size of chromaticity residual samples Cr , By performing this process, a residual sample array of size (nTbW)×(nTbH) is generated. Y r Cb and r Cr The following updates are possible.

[0825] First, when the value of sps_act_reversible_conversion is the second value (e.g., 1), the residual sample array r of size (nTbW)×(nTbH) Y r Cb and r Cr It can be updated as shown in the following formula.

[0826] [Equation 112]

[0827] tmp=r Y [x][y]-(r Cb [x][y]>>1))

[0828] r Y [x][y]=tmp+r Cb [x][y])

[0829] r Cb [x][y]=tmp-(r Cr [x][y]>>1))

[0830] r Cr [x][y]=r Cb [x][y]+r Cr [x][y]

[0831] Otherwise (e.g., when the value of sps_act_reversible_conversion is the first value (e.g., 0)), the residual sample array r of size (nTbW)×(nTbH) Y r Cb and r Cr It can be updated as shown in the following formula.

[0832] [Equation 113]

[0833] tmp=r Y [x][y]-r Cb [x][y]

[0834] r Y [x][y]=r Y [x][y]+r Cb [x][y]

[0835] rCb [x][y]=tmp-r Cr [x][y]

[0836] r Cr [x][y]=tmp+r Cr [x][y]

[0837] The YCgCo backward transformation and the YCgCo-R backward transformation share some similarities. In a reversible transformation back to the original state, this can be considered a lossy backward transformation operation when Cg and Co are replaced with Cg'=Cg<<1 and Co'=Co<<1. The following equation illustrates its implementation.

[0838] [Equation 114]

[0839] t = Y - (Cg' >> 1) = Y - Cg

[0840] G=Cg'+t=Y+Cg

[0841] B = t - (Co' >> 1) = t - Co = Y - Cg - Co

[0842] R = Co' + B = t + Co = Y - Cg + Co

[0843] Therefore, in an alternative implementation, instead of maintaining two color transformations, only a reversible transformation to the original state can be used. In the case of lossy encoding, the Cg and Co components can be scaled by a factor of 1 / 2 during the operation of the encoding device and by a factor of 2 during the operation of the decoding device. This allows for the use of an integrated transformation even when both lossy and lossless cases are supported. Additionally, there is the added advantage that the bit depth remains unchanged even when lossy encoding is in progress.

[0844] [Table 7]

[0845] In the implementation method, it can be based on Figure 51 The syntax uses flags that indicate which ACT transformation to use (e.g., actShiftFlag). Figure 51In the syntax table, the syntax element `sps_act_shift_flag` indicates whether color component shifting is performed concurrently with the application of ACT. For example, a first value of `sps_act_shift_flag` (e.g., 0) indicates that color component shifting is not performed concurrently with the application of ACT. A second value of `sps_act_shift_flag` (e.g., 1) indicates that color component shifting is performed concurrently with the application of ACT. The variable `actShiftFlag` can be set to the value of `sps_act_shift_flag`. The pseudocode for implementing a backward conversion from YcgCo to GBR in a decoding device can be written using `actShiftFlag` as follows.

[0846] [Table 8]

[0847] Implementation Method 11: ACT Execution Method for Performing Multiple Color Transformations Using Conversion Type Derivation

[0848] In implementation, at least one color conversion can be used when performing ACT. Furthermore, the type of color conversion to use can be deduced based on other information in the bitstream.

[0849] In the implementation, two types of ACT transformations are available: reversible ACT transformations to the original state and irreversible ACT transformations to the original state. The ACT transformation type can be derived from the transformation type. For example, as identified by the variable tuIsTransformSkip, a reversible ACT transformation to the original state can be used when the transformation type is transformation skip. Otherwise (e.g., when the transformation type is not transformation skip), an irreversible ACT transformation to the original state can be used. Two types of pseudocode can be used.

[0850] [Table 9]

[0851] [Table 10]

[0852] In another implementation, the ACT transition type can be determined based on the QP value. When the Qp value is less than or equal to a predetermined threshold (e.g., QpPrimeTsMin), a reversible ACT transition to the original state can be used. Otherwise (e.g., when the Qp value exceeds the predetermined threshold), an irreversible ACT transition can be used.

[0853] Implementation Method 12: QP Derivation Method Using ACT QP Offset

[0854] This implementation method relates to Implementation Method 1 and Implementation Method 2 described above. In Implementation Method 1 and Implementation Method 2, the derived Qp' has already been described.Y Qp' CbCr Qp' Cb and Qp' Cr Included as QP. The methods described in Embodiments 1 and 2 use ACT QP offset to correct the derived Qp value, and apply necessary clipping techniques to ensure that the corrected QP value for scaling the transformation coefficient does not exceed the effective range.

[0855] This embodiment describes the method used for deriving Qp' Y Qp' CbCr Qp' Cb and Qp' Cr The QP derivation process includes an ACTQP offset method. The QP derivation process already includes a predetermined trimming step to ensure that the derived QP value does not exceed the valid range. Therefore, including the ACTQP offset in the QP derivation process avoids additional trimming steps, simplifies the overall QP derivation process for transformation coefficient scaling, and ensures that the final QP does not exceed the valid range.

[0856] As described in the above embodiments, the ACT QP offset can be pre-specified as a constant or signaled via the bitstream. Without loss of consistency, the ACT QP offsets of Y, Cb, Cr, and CbCr can be described later as ppsActQpOffsetY, ppsActQpOffsetCb, ppsActQpOffsetCr, and ppsActQpOffsetCbCr. ppsActQpOffsetY, ppsActQpOffsetCb, ppsActQpOffsetCr, and ppsActQpOffsetCbCr can be constants or variables with values ​​ranging from -M to N. In this document, in the embodiments, each of M and N can be set to 12 in the case of lossy encoding and to 0 in the case of lossless encoding. Furthermore, at least one ACT QP offset can be derived from another ACT QP offset value. For example, ppsActQpOffsetCbCr can be set to the same value as ppsActQpOffsetCb or ppsActQpOffsetCr based on the jointCbCr mode.

[0857] Decoding processing using QP derivation with ACT QP offset can be performed as follows. First, in the case of quantization parameter derivation processing, the following parameters can be used in this process.

[0858] - Luminosity coordinates (xCb, yCb), relative to the top-left luminosity sample of the current frame, indicating the relative coordinates of the top-left luminosity sample of the current coded block. - The variable cbWidth indicates the width of the current coded block based on each luminance sample. - Variable cbHeight, indicating the height of the current coding block based on each luminance sample. - The variable treeType indicates whether to use a single tree (SINGLE_TREE) or a dual tree to split the current coding tree node, and when using a dual tree, it indicates whether the dual tree is a luma component dual tree (DAUL_TREE_LUMA) or a chroma component dual tree (DAUL_TREE_CHROMA).

[0859] In this quantization parameter derivation process, the brightness quantization parameter Qp' can be derived. Y and colorimetric parameter Qp' Cb Qp' Cr and Qp' CbCr .

[0860] After that, variable Qp Y It can be derived as shown in the following formula.

[0861] [Formula 115]

[0862] Qp Y =((qP Y_PRED +CuQpDeltaVal+64+2 QpBdOffset)%(64+QpBdOffset))-

[0863] QpBdOffset

[0864] Brightness quantization parameter Qp' Y It can be derived as shown in the following formula.

[0865] [Equation 116]

[0866] actQpOffsetY=cu_act_enabled_flag[xCb][yCb] ? ppsActQpOffsetY:0

[0867] Qp' Y =Qp Y +QpBdOffset+actQpOffsetY

[0868] Qp' Y =Clip3(0, 63+QpBdOffset, Qp' Y )

[0869] When the value of the variable ChromaArrayType, which indicates the type of the chroma array, is not the first value (e.g., 0) and treeType is SINGLE_TREE or DUAL_TREE_CHROMA, the following processing can be performed.

[0870] - When the value of treeType is DUAL_TREE_CHROMA, the variable Qp Y The value can be set to the luminance quantization parameter Qp based on the luminance coding at the luminance sample location (xCb+cbWidth / 2, yCb+cbHeight / 2). Y The value is the same as the value.

[0871] -Variable qP Cb qP Cr and qP CbCr It can be derived as shown in the following formula.

[0872] [Formula 117]

[0873] qP Chroma =Clip3(-QpBdOffset, 63, Qp Y )

[0874] qP Cb =ChromaQpTable[0][qP Chroma ]

[0875] qP Cr =ChromaQpTable[1][qP Chroma ]

[0876] qP CbCr =ChromaQpTable[2][qP Chroma ]

[0877] - The colorimetric parameters Qp' for the Cb and Cr components Cb and Qp' Cr and the colorimetric parameter Qp' of the joint Cb-Cr encoding CbCr It can be derived as shown in the following formula.

[0878] [Formula 118]

[0879] actQpOffsetCb=cu_act_enabled_flag[xCb][yCb] ? ppsActQpOffsetCb:0

[0880] actQpOffsetCr=cu_act_enabled_flag[xCb][yCb] ? ppsActQpOffsetCr:0

[0881] actQpOffsetCbCr=cu_act_enabled_flag[xCb][yCb] ? ppsActQpOffsetCbCr:0

[0882] Qp' Cb =Clip3(-QpBdOffset, 63, qP Cb +pps_cb_qp_offset+slice_cb_qp_offset+CuQpOffset Cb +actQpOffsetCb)+QpBdOffset

[0883] Qp' Cr =Clip3(-QpBdOffset, 63, qP Cr +pps_cr_qp_offset+slice_cr_qp_offset+CuQpOffset Cr +actQpOffsetCr)+QpBdOffset

[0884] Qp' CbCr =Clip3(-QpBdOffset, 63, qP CbCr +pps_joint_cbcr_qp_offset+slice_joint_cbcr _qp_offset+CuQpOffset CbCr +actQpOffsetCbCr)+QpBdOffset

[0885] Next, the dequantization of the transform coefficients can be performed, and the following information can be used as input for this process.

[0886] - The luminance coordinates (xTbY, yTbY) refer to the relative coordinates of the top-left sample of the current luminance transform block with respect to the top-left luminance sample of the current image. - The variable nTbW indicates the width of the transform block. - The variable nTbH indicates the height of the transform block. - The variable predMode indicates the prediction mode underlying the encoding. - The variable cIdx indicates the color components of the current block. The output of the dequantization of the transform coefficients can be an array d of scaled transform coefficients. In this paper, the size of the array d can be (nTbW)×(nTbH). The individual elements constituting this array can be labeled d[x][y].

[0887] When performing this process, the quantization parameter qP can be derived as follows. When cIdx has a value of 0, qP can be derived as shown in the following equation.

[0888] [Equation 119]

[0889] qP=Qp' Y

[0890] Alternatively, when TuCResMode[xTbY][yTbY] has a value of 2, qP can be derived as shown in the following equation.

[0891] [Formula 120]

[0892] qP=Qp' CbCr

[0893] Alternatively, when cIdx has a value of 1, qP can be derived as shown in the following equation.

[0894] [Equation 121]

[0895] qP=Qp' Cb

[0896] Alternatively, when cIdx has a value of 2, qP can be derived as shown in the following equation.

[0897] [Equation 122]

[0898] qP=Qp' Cr

[0899] The quantization parameter qP can be updated as follows. Additionally, the variables rectNonTsFlag and bdShift can be derived as follows. When transform_skip_flag[xTbY][yTbY][cIdx] has a value of 0, the variables can be derived as shown in the following equation.

[0900] [Equation 123]

[0901] rectNonTsFlag=(((Log2(nTbW)+Log2(nTbH)) & 1)==1) ? 1:0

[0902] bdShift=BitDepth+rectNonTsFlag+((Log2(nTbW)+Log2(nTbH)) / 2)-5+pic_dep_quant_enabled_flag

[0903] Alternatively, when transform_skip_flag[xTbY][yTbY][cIdx] has a value of 1 (e.g., the transformation of the current transform block is skipped), the parameters and variables can be derived as shown in the following equation.

[0904] [Equation 124]

[0905] qP=Max(QpPrimeTsMin, qP)

[0906] rectNonTsFlag=0

[0907] bdShift=10

[0908] Encoding methods and decoding methods

[0909] In the following text, reference will be made to Figure 52 and Figure 53 This describes an image encoding method performed by an image encoding device and an image decoding method performed by an image decoding device.

[0910] First, the operation of the decoding device will be described. The image decoding device according to an embodiment may include a memory and a processor. The decoding device can perform decoding according to the operation of the processor. For example, as... Figure 52 As shown, in step S5210, the decoding device can determine the quantization parameters of the current block based on whether a color space conversion is applied to the residual samples of the current block. In this document, the color space conversion can be the aforementioned ACT. In this document, the determination of the quantization parameters can be performed by pruning the quantization parameters so that the values ​​of the quantization parameters have values ​​less than or equal to a predetermined upper limit and greater than or equal to a predetermined lower limit. In this document, the predetermined lower limit of the quantization parameters can be 0. Furthermore, the predetermined upper limit of the quantization parameters can be determined based on syntax elements indicating the bit depth of the samples.

[0911] Determining the quantization parameters may include: determining the quantization parameters based on the color components of the current block; determining the quantization parameter offset based on the color components of the current block; and using the quantization parameter offset to reset the quantization parameters.

[0912] Resetting the quantization parameter using a quantization parameter offset can be performed by adding the quantization parameter offset to the quantization parameter. In this paper, when a color space conversion is applied to the residual sample of the current block and the color component of the current block is a luminance component, the value of the quantization parameter offset can be determined to be -5.

[0913] Additionally, when a color space transformation is applied to the residual sample of the current block and the color component of the current block is the chromaticity Cb component, the quantization parameter offset value can be determined to be 1. When a color space transformation is applied to the residual sample of the current block and the color component of the current block is the chromaticity Cr component, the quantization parameter offset value can be determined to be 3.

[0914] Next, in step S5220, the decoding device can determine the transform coefficients of the current block based on the quantization parameters. Next, in step S5230, the decoding device can use the transform coefficients to determine the residual samples of the current block. Next, in step S5240, the decoding device can reset the values ​​of the residual samples based on whether a color space conversion is applied.

[0915] In this paper, the values ​​of residual samples can be reset based on the half values ​​of the luminance component residual sample values ​​and the chrominance component residual sample values. For example, the half value of the chrominance residual sample value can be obtained by performing a shift operation on the chrominance residual sample value. Alternatively, the half value of the chrominance Cb component residual sample value can be added to the luminance component residual sample value to reset the luminance component residual sample value. Furthermore, the half values ​​of the chrominance Cb component residual sample value and the half values ​​of the chrominance Cr component residual sample value can be subtracted from the luminance component residual sample value to reset the chrominance Cb component residual sample value.

[0916] The operation of the encoding device will be described below. The image encoding device according to an embodiment may include a memory and a processor. The encoding device can perform encoding according to the operation of the processor in a manner corresponding to decoding by the decoding device. For example, as Figure 53 As shown, in step S5310, the encoding device can reset the residual sample based on whether a color space conversion is applied. In this document, the color space conversion can be the aforementioned ACT.

[0917] Furthermore, the values ​​of the residual samples can be reset based on the half values ​​of the luminance component residual sample values ​​and the chrominance residual sample values. For example, the half value of the chrominance residual sample value can be obtained by performing a shift operation on the chrominance residual sample value. In addition, by performing the reset operation in reverse by the decoding device described above, the encoding device can reset the luminance component residual sample values, the chrominance Cb component residual sample values, and the chrominance Cr component residual sample values.

[0918] Next, in step S5320, the encoding device can determine the transform coefficients using the reset residual samples. Next, in step S5330, the encoding device can determine the quantization parameters based on whether a color space conversion is applied. In this document, the determination of the quantization parameters can be performed by pruning the quantization parameters so that the values ​​of the quantization parameters have values ​​less than or equal to a predetermined upper limit and greater than or equal to a predetermined lower limit. In this document, the predetermined lower limit of the quantization parameters can be 0. Alternatively, the predetermined upper limit of the quantization parameters can be determined based on syntax elements indicating the bit depth of the samples.

[0919] Determining the quantization parameters may include: determining the quantization parameters based on the color components of the current block; determining the quantization parameter offset based on the color components of the current block; and using the quantization parameter offset to reset the quantization parameters.

[0920] Resetting the quantization parameter using a quantization parameter offset can be performed by adding the quantization parameter offset to the quantization parameter. In this paper, when a color space conversion is applied to the residual sample of the current block and the color component of the current block is a luminance component, the value of the quantization parameter offset can be determined to be -5.

[0921] Additionally, when a color space transformation is applied to the residual sample of the current block and the color component of the current block is the chromaticity Cb component, the quantization parameter offset value can be determined to be 1. When a color space transformation is applied to the residual sample of the current block and the color component of the current block is the chromaticity Cr component, the quantization parameter offset value can be determined to be 3.

[0922] Next, in step S5340, the encoding device can encode the transform coefficients based on the quantization parameters.

[0923] Application and Implementation Methods

[0924] Although the exemplary methods of this disclosure described above are represented as a series of operations for clarity of description, they are not intended to limit the order in which the steps are performed, and these steps may be performed simultaneously or in different orders if necessary. To implement the method according to the invention, the described steps may further include other steps, including steps in addition to some steps, or may include additional steps in addition to some steps.

[0925] In this disclosure, the image encoding device or image decoding device that performs a predetermined operation (step) can perform an operation (step) that confirms the execution conditions or circumstances of the corresponding operation (step). For example, if it is described that a predetermined operation is performed when predetermined conditions are met, the image encoding device or image decoding device can perform the predetermined operation after determining whether the predetermined conditions are met.

[0926] The various embodiments of this disclosure are not a list of all possible combinations and are intended to describe representative aspects of this disclosure; the matters described in the various embodiments may be applied independently or in combination of two or more.

[0927] Various embodiments of this disclosure can be implemented in hardware, firmware, software, or a combination thereof. When this disclosure is implemented in hardware, it can be implemented using application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, etc.

[0928] Furthermore, the image decoding and image encoding devices applying the embodiments of this disclosure can be included in multimedia broadcasting transmission and receiving devices, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video chat devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, video-on-demand (VoD) service providers, OTT (over-the-top) video devices, internet streaming service providers, three-dimensional (3D) video devices, video telephony devices, medical video devices, etc., and can be used to process video signals or data signals. For example, OTT video devices can include game consoles, Blu-ray players, internet access televisions, home theater systems, smartphones, tablet PCs, digital video recorders (DVRs), etc.

[0929] Figure 54 This is a view illustrating a content streaming system to which embodiments of the present disclosure can be applied.

[0930] like Figure 54 As shown, the content streaming system applying the embodiments of this disclosure may mainly include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[0931] The encoding server compresses content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream and then sends the bitstream to the streaming server. As another example, when multimedia input devices such as smartphones, cameras, and camcorders directly generate bitstreams, the encoding server can be omitted.

[0932] The bitstream can be generated by an image encoding method or image encoding device applying the embodiments of this disclosure, and the stream server can temporarily store the bitstream during the sending or receiving of the bitstream.

[0933] A streaming server sends multimedia data to a user's device based on a request from a web server, and the web server acts as a medium for informing the user of the service. When a user requests a service from the web server, the web server can deliver it to the streaming server, and the streaming server can send the multimedia data to the user. In this scenario, the content streaming system may include a separate control server. In this case, the control server is used to control the commands / responses between devices in the content streaming system.

[0934] A streaming server can receive content from media storage devices and / or encoding servers. For example, when receiving content from an encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a predetermined period of time.

[0935] Examples of user devices may include mobile phones, smartphones, laptops, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head-mounted displays), digital televisions, desktop computers, digital signage, etc.

[0936] In a content streaming system, each server can operate as a distributed server, in which case the data received from each server can be distributed.

[0937] The scope of this disclosure includes software or machine-executable commands (e.g., operating systems, applications, firmware, programs, etc.) for enabling the operation of methods according to various embodiments to be executed on a device or computer, and non-transitory computer-readable media having such software or commands stored thereon and executable on a device or computer.

[0938] Industrial applicability

[0939] The embodiments disclosed herein can be used to encode or decode images.

Claims

1. An image decoding method performed by an image decoding device, the image decoding method comprising the following steps: The current block is determined by segmenting the image. Determine the tree type of the current block; The quantization parameters of the current block are derived based on the color space transformation of the residual samples of the current block; The transformation coefficients of the current block are derived based on the quantization parameters; The residual sample of the current block is derived based on the transformation coefficients; as well as Based on the color space transformation described above, the values ​​of the residual samples are modified. Specifically, the quantization parameters are derived through a pruning operation based on the value of the transform skip flag of the current block. The cropping operation includes adjusting the value of the quantization parameter to a range from a predetermined lower limit to a predetermined upper limit, wherein the range includes the predetermined lower limit and the predetermined upper limit. The predetermined upper limit value is derived based on the sum of the quantization parameter range offset and the predetermined constant value. Specifically, the decision to apply the color space transformation to the residual samples of the current block is determined based on the tree type of the current block. Where the tree type of the current block is not a single tree, the color space transformation is not applied. Wherein, based on the transform skip flag indicating that the transform is not applied to the current block, the predetermined lower limit value is derived as the minimum allowable quantization parameter for the transform skip mode, and The application of the transformation to the current block based on the transformation skip flag is determined by other syntax elements, and the predetermined lower limit value is derived to be 0.

2. The image decoding method according to claim 1, in, The predetermined constant value is 63.

3. The image decoding method according to claim 1, in, The quantization parameters are derived based on the initial quantization parameters derived from the color components of the current block.

4. The image decoding method according to claim 1, in, The quantization parameters are derived based on the quantization parameter offset derived from the color components of the current block.

5. The image decoding method according to claim 4, in, Based on the fact that the color component of the current block is a luminance component and the application of the color space transformation, the magnitude of the quantization parameter offset is derived to be -5.

6. The image decoding method according to claim 4, in, Based on the fact that the color component of the current block is a chromaticity Cr component and the application of the color space transformation, the magnitude of the quantization parameter offset is derived to be 3.

7. The image decoding method according to claim 4, in, The subtraction result of the first value of the quantization parameter offset and the second value of the quantization parameter offset is 2. Specifically, based on the color space transformation, the first value is derived by shifting the quantization parameter of the color component of the current block to the chromaticity Cr component, and the second value is derived by shifting the quantization parameter of the color component of the current block to the chromaticity Cb component.

8. The image decoding method according to claim 1, in, The predetermined lower limit value is derived based on the transformation skip flag of the current block.

9. The image decoding method according to claim 1, in, Based on the transform skip flag indicating that the transform is not applied to the current block, the predetermined lower limit is derived as the minimum permissible quantization parameter for the transform skip mode.

10. The image decoding method according to claim 1, in, The information regarding the application of the transformation indicates that the transformation is applied to the current block, which is determined by other syntax elements, and the predetermined lower limit value is derived to be 0.

11. The image decoding method according to claim 1, in, The value of the quantization parameter range offset is derived based on the syntax elements for the bit depth of the current block.

12. An image encoding method performed by an image encoding device, the image encoding method comprising the following steps: The current block is determined by segmenting the image. Determine the tree type of the current block; Based on the applied color space transformation, the value of the residual sample of the current block is derived; The transformation coefficients of the current block are derived based on the residual samples of the current block; Based on the applied color space transformation, the quantization parameters of the current block are determined; and The transform coefficients are encoded based on the quantization parameters. Specifically, the quantization parameters are derived through a pruning operation based on the value of the transform skip flag of the current block. The cropping operation includes adjusting the value of the quantization parameter to a range from a predetermined lower limit to a predetermined upper limit, wherein the range includes the predetermined lower limit and the predetermined upper limit. The predetermined upper limit value is derived based on the sum of the quantization parameter range offset and the predetermined constant value. Specifically, the decision to apply the color space transformation to the residual samples of the current block is based on the tree type of the current block. Where the tree type of the current block is not a single tree, the color space transformation is not applied. Wherein, the value of the transform skip flag is determined to be 1, and the predetermined lower limit value is derived as the minimum allowable quantization parameter for the transform skip mode, and Specifically, the value of the transformation skip flag is determined to be 0, and the predetermined lower limit value is derived to be 0.

13. The image encoding method according to claim 12, in, The predetermined constant value is 63.

14. A method for transmitting a bit stream, the method comprising the following steps: Perform the image encoding method according to claim 12 to generate a bitstream; and Send the bit stream.

Images