Image decoding / encoding method, method for storing bitstream, and method for transmitting data

Abstract

The present disclosure relates to an image decoding / encoding method, a method of storing a bitstream, and a method of transmitting data. According to the disclosure of the present document, a chroma residual scaling factor is derived based on information related to an absolute value of a chroma residual scaling delta value and information related to a symbol of the chroma residual scaling delta value, and an LMCS process may be efficiently performed by applying explicit signaling to the information related to a value of the chroma residual scaling factor.

Description

[0001] This application is a divisional application of the original invention patent application with application number 202080060720.0 (international application number: PCT / KR2020 / 008808, application date: July 6, 2020, invention name: Video or image coding based on brightness mapping and chroma scaling). Technical Field

[0002] The technology of this document relates to video or image coding based on luma mapping and chroma scaling. Background Art

[0003] Recently, the demand for high-resolution, high-quality images / videos such as 4K or 8K or higher ultra-high-definition (UHD) images / videos has increased in various fields. As image / video data has high resolution and high quality, the amount of information or the amount of bits to be transmitted increases relative to existing image / video data, and therefore, transmitting image data using a medium such as an existing wired / wireless broadband line or an existing storage medium or storing image / video data using an existing storage medium increases transmission costs and storage costs.

[0004] In addition, interest in and demand for immersive media such as virtual reality (VR) and artificial reality (AR) content or holograms have recently increased, and broadcasting of images / videos having characteristics different from real images (e.g., game images) has increased.

[0005] Therefore, very efficient image / video compression technology is required to effectively compress, transmit, store and reproduce information of high-resolution, high-quality images / videos having various characteristics as described above.

[0006] Furthermore, to improve compression efficiency and increase subjective / objective visual quality, a Luma Mapping and Chroma Scaling (LMCS) process is performed, and increasing the accuracy in the LMCS process is discussed. Summary of the invention

[0007] Technical Solution

[0008] According to an embodiment of this document, a method and apparatus for increasing image encoding efficiency are provided.

[0009] According to the embodiments of this document, a high-efficiency filtering application method and device are provided.

[0010] According to the embodiments of this document, an efficient LMCS application method and device are provided.

[0011] According to an embodiment of this document, a LMCS codeword (or a range thereof) may be constrained.

[0012] According to embodiments of this document, a single chroma residual scaling factor that is directly signaled in the chroma scaling of the LMCS may be used.

[0013] According to an embodiment of the present document, a linear mapping (linear LMCS) may be used.

[0014] According to an embodiment of the present document, information about the pivot point required for the linear mapping may be explicitly signaled.

[0015] According to embodiments of this document, a flexible number of bins may be used for brightness mapping.

[0016] According to the embodiments of this document, the index derivation process for inverse luma mapping and / or chroma residual scaling can be simplified.

[0017] According to an embodiment of the present document, the LMCS process may be applied even when a luma block and a chroma block in one coding tree unit (CTU) have separate block tree structures (dual tree structures).

[0018] According to embodiments of this document, information about the value of the chroma residual scaling factor may be explicitly signaled or implicitly derived.

[0019] According to an embodiment of the present document, explicit signaling and / or implicit derivation may be selectively switched for information on the value of the chroma residual scaling factor.

[0020] According to an embodiment of this document, a video / image decoding method performed by a decoding device is provided.

[0021] According to an embodiment of this document, a decoding device for performing video / image decoding is provided.

[0022] According to an embodiment of this document, a video / image encoding method performed by an encoding device is provided.

[0023] According to an embodiment of this document, there is provided an encoding device for performing video / image encoding.

[0024] According to one embodiment of the present document, a computer-readable digital storage medium is provided, in which encoded video / image information generated according to the video / image encoding method disclosed in at least one embodiment of the present document is stored.

[0025] According to an embodiment of the present document, a computer-readable digital storage medium is provided, in which encoded information or encoded video / image information is stored for enabling a decoding device to execute a video / image decoding method disclosed in at least one embodiment of the present document.

[0026] Beneficial Effects

[0027] According to the embodiments of this document, the overall image / video compression efficiency can be improved.

[0028] According to the embodiments of this document, subjective / objective visual quality can be improved through efficient filtering.

[0029] According to the embodiments of this document, LMCS processing for image / video encoding can be efficiently performed.

[0030] According to the embodiments of this document, the resources / costs (software or hardware) required for LMCS processing can be minimized.

[0031] According to the implementation of this document, hardware implementation of LMCS processing can be facilitated.

[0032] According to embodiments of the present document, the division operations required for the derivation of LMCS codewords in the mapping (shaping) can be removed or minimized by constraining the LMCS codewords (or their range).

[0033] According to an embodiment of the present document, a single chroma residual scaling factor may be used to remove the delay identified according to the segment index.

[0034] According to an embodiment of the present document, a chroma residual scaling process can be performed using a linear mapping in LMCS without relying on (reconstruction of) luma blocks, thus removing the delay in scaling.

[0035] According to the embodiments of this document, mapping efficiency in LMCS can be increased.

[0036] According to the embodiments of the present document, by simplifying the index derivation process for inverse luma mapping and / or chroma residual scaling, the complexity of LMCS can be reduced, and thus the video / image coding efficiency can be increased.

[0037] According to the embodiment of the present document, the LMCS process can be performed even on a block having a dual tree structure, so the efficiency of LMCS can be increased. In addition, the encoding performance (eg, objective / subjective image quality) of a block having a dual tree structure can be improved.

[0038] According to the embodiments of this document, since explicit signaling or implicit derivation is selectively applied to information about the value of the chroma residual scaling factor, the LMCS process can be efficiently performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 An example of a video / image encoding system to which embodiments of the present disclosure can be applied is schematically shown.

[0040] Figure 2is a diagram schematically showing a configuration of a video / image encoding device to which an embodiment of the present disclosure can be applied.

[0041] Figure 3 is a diagram schematically showing a configuration of a video / image decoding device to which an embodiment of the present disclosure can be applied.

[0042] Figure 4 An exemplary block tree architecture is shown.

[0043] Figure 5 The hierarchical structure of the encoded video / image is shown exemplarily.

[0044] Figure 6 The layered architecture of the CVS according to the embodiment of this document is exemplarily shown.

[0045] Figure 7 The layered architecture of the CVS according to the embodiment of this document is exemplarily shown.

[0046] Figure 8 The layered architecture of the CVS according to another embodiment of the present document is exemplarily shown.

[0047] Fig. 9 An exemplary LMCS architecture according to an embodiment of this document is shown.

[0048] Fig.10 An LMCS architecture according to another embodiment of the present document is shown.

[0049] Fig.11 A graph representing an exemplary forward map is shown.

[0050] Fig.12 is a flowchart illustrating a method of deriving a chroma residual scaling index according to an embodiment of this document.

[0051] Fig.13 A linear fit of the pivot points according to an embodiment of the present document is shown.

[0052] Fig.14 An example of a linear shaper according to an embodiment of the present document is shown.

[0053] Fig.15 An example of linear forward mapping in an embodiment of this document is shown.

[0054] Fig.16 An example of reverse forward mapping in the embodiment of this document is shown.

[0055] Fig.17 is a diagram illustrating a LMCS process according to an embodiment of the present document.

[0056] Fig.18 and Fig.19 An example of a video / image encoding method and related components according to an embodiment of the present document are schematically shown.

[0057] Fig. 20 and Fig.21 An example of an image / video decoding method and related components according to an embodiment of this document are schematically shown.

[0058] Fig. 22 An example of a content streaming system to which the embodiments disclosed in this document can be applied is shown. DETAILED DESCRIPTION

[0059] This document can be modified in various forms, and its specific embodiments will be described and shown in the drawings. However, these embodiments are not intended to limit this document. The terms used in the following description are only used to describe specific embodiments and are not intended to limit this document. Singular expressions include plural expressions as long as they are clearly read differently. Terms such as "including" and "having" are intended to indicate the presence of features, quantities, steps, operations, elements, components, or combinations thereof used in the following description, so it should be understood that the possibility of the presence or addition of one or more different features, quantities, steps, operations, elements, components, or combinations thereof is not excluded.

[0060] In addition, each configuration in the drawings described in this document is shown independently for the convenience of describing different characteristic functions, and does not mean that each configuration is implemented as separate hardware or separate software. For example, two or more components among each component can be combined to form a component, or a component can be divided into multiple components. Embodiments in which each component is integrated and / or separated are also included in the disclosure scope of this document.

[0061] Hereinafter, examples of the present embodiment will be described in detail with reference to the accompanying drawings. In addition, like reference numerals are used to indicate like elements throughout the drawings, and the same description about the like elements will be omitted.

[0062] Figure 1 An example of a video / image encoding system to which the embodiments of this document can be applied is shown.

[0063] Reference Figure 1 The video / image coding system may include a first device (source device) and a second device (receiving device). The source device may send the coded video / image information or data in the form of a file or stream to the receiving device via a digital storage medium or a network.

[0064] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display, and the display may be configured as a separate device or an external component.

[0065] The video source may acquire the video / image by capturing, synthesizing or generating the video / image process. The video source may include a video / image capture device and / or a video / image generation device. For example, the video / image capture device may include one or more cameras, a video / image archive including previously captured videos / images, etc. For example, the video / image generation device may include a computer, a tablet computer, and a smart phone, and may generate the video / image (electronically). For example, a virtual video / image may be generated by a computer, etc. In this case, the video / image capture process may be replaced by a process that generates relevant data.

[0066] The encoding device can encode the input video / image. For compression and encoding efficiency, the encoding device can perform a series of processes such as prediction, transformation and quantization. The encoded data (encoded video / image information) can be output in the form of a bit stream.

[0067] The transmitter may transmit the encoded image / image information or data output in the form of a bit stream to a receiver of a receiving device in the form of a file or stream through a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter may include an element for generating a media file in a predetermined file format, and may include an element for transmission through a broadcast / communication network. The receiver may receive / extract a bit stream and transmit the received bit stream to a decoding device.

[0068] The decoding device may decode a video / image by performing a series of processes corresponding to the operations of the encoding device, such as dequantization, inverse transformation, and prediction.

[0069] The renderer may render the decoded video / image. The rendered video / image may be displayed through a display.

[0070] This document relates to video / image coding. For example, the methods / implementations disclosed in this document may be applied to methods disclosed in the Versatile Video Coding (VVC) standard, the Essential Video Coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the 2nd generation Audio Video Coding standard (AVS2), or the next generation video / image coding standard (e.g., H.267, H.268, etc.).

[0071] This document proposes various embodiments of video / image coding, and unless otherwise specified, the above embodiments may also be performed in combination with each other.

[0072] In this document, video may refer to a series of images over time. A picture generally refers to a unit representing an image of a specific time range, and a slice / tile refers to a unit that constitutes a part of a picture in terms of coding. A slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / tiles. A picture may be composed of one or more tile groups. A tile group may include one or more tiles. A tile may represent a rectangular area of ​​a CTU row within a tile in a picture. A tile may be divided into a plurality of tiles, each tile may be composed of one or more CTU rows within a tile. Tiles that are not divided into a plurality of tiles may also be referred to as tiles. Tile scanning may represent a specific sequential ordering of CTUs of a partitioned picture, wherein CTUs may be ordered in a raster scan of CTUs within tiles, and tiles within tiles may be ordered continuously in a raster scan of tiles of tiles, and tiles in a picture may be ordered continuously in a raster scan of tiles of a picture. A tile is a rectangular area of ​​CTUs within a specific tile column and a specific tile row in a picture. A tile column is a rectangular area of ​​a CTU whose height is equal to the height of the picture and whose width is specified by a syntax element in the picture parameter set. A tile row is a rectangular area of ​​a CTU whose height is specified by a syntax element in the picture parameter set and whose width is equal to the width of the picture. A tile scan is a specific sequential ordering of the CTUs of a partitioned picture, where the CTUs are ordered continuously in a CTU raster scan of the tiles, and the tiles in the picture are ordered continuously in a raster scan of the tiles of the picture. A slice includes an integer number of tiles of a picture that can be exclusively contained in a single NAL unit. A slice may consist of a continuous sequence of multiple complete tiles or only complete tiles of one tile. In this document, tile groups and slices may be used instead of each other. For example, in this document, a tile group / tile group header may be referred to as a slice / slice header.

[0073] Furthermore, a picture may be divided into two or more sub-pictures. A sub-picture may be a rectangular area of ​​one or more slices within a picture.

[0074] A pixel or picture element may refer to the smallest unit constituting a picture (or image). In addition, a "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component.

[0075] A unit may represent a basic unit of image processing. A unit may include at least one of a specific area of ​​a picture and information related to the area. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, a unit may be used interchangeably with terms such as a block or an area. In general, an M×N block may include M columns and N rows of samples (or sample arrays) or a set (or array) of transform coefficients. Alternatively, a sample may refer to a pixel value in a spatial domain, and when such a pixel value is transformed to a frequency domain, it may refer to a transform coefficient in a frequency domain.

[0076] In this document, "A or B" may mean "only A", "only B", or "both A and B". In other words, "A or B" in this document may be interpreted as "A and / or B". For example, in this document, "A, B or C (A, B or C)" means "only A", "only B", "only C", or "any combination of A, B, and C".

[0077] A slash ( / ) or a comma (,) as used in this document may mean "and / or". For example, "A / B" may mean "A and / or B". Thus, "A / B" may mean "only A", "only B", or "both A and B". For example, "A, B, C" may mean "A, B, or C".

[0078] In this document, "at least one of A and B" may mean "only A", "only B", or "both A and B". In addition, in this document, the expression "at least one of A or B" or "at least one of A and / or B" may be interpreted the same as "at least one of A and B".

[0079] In addition, in this document, "at least one of A, B, and C" means "only A", "only B", "only C", or "any combination of A, B, and C". In addition, "at least one of A, B, or C" or "at least one of A, B and / or C" may mean "at least one of A, B, and C".

[0080] In addition, brackets used in this document may mean "for example". Specifically, when "prediction (intra-frame prediction)" is indicated, "intra-frame prediction" may be proposed as an example of "prediction". In other words, "prediction" in this document is not limited to "intra-frame prediction", and "intra-frame prediction" may be proposed as an example of "prediction". In addition, even when "prediction (i.e., intra-frame prediction)" is indicated, "intra-frame prediction" may be proposed as an example of "prediction".

[0081] Technical features described individually in one figure in this document may be implemented individually or simultaneously.

[0082] Figure 2Schematically shows the configuration of a video / image encoding device to which an embodiment of the present document can be applied. Hereinafter, the so-called video encoding device may include an image encoding device.

[0083] Reference Figure 2 , the encoding device 200 includes an image segmenter 210, a predictor 220, a residual processor 230 and an entropy encoder 240, an adder 250, a filter 260 and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234 and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. According to an embodiment, the image segmenter 210, the predictor 220, the residual processor 230, the entropy encoder 240, the adder 250 and the filter 260 may be configured by at least one hardware component (e.g., an encoder chipset or processor). In addition, the memory 270 may include a decoded picture buffer (DPB), or may be configured by a digital storage medium. The hardware component may also include the memory 270 as an internal / external component.

[0084] The image divider 210 may divide the input image (or picture or frame) input to the encoding device 200 into one or more processors. For example, the processor may be referred to as a coding unit (CU). In this case, the coding unit may be recursively divided from a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree binary tree ternary tree (QTBTTT) structure. For example, one coding unit may be divided into a plurality of coding units with a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary structure. In this case, for example, a quadtree structure may be applied first, and a binary tree structure and / or a ternary structure may be applied later. Alternatively, a binary tree structure may be applied first. The encoding process according to the present disclosure may be performed based on a final coding unit that is no longer divided. In this case, the maximum coding unit may be used as the final coding unit based on coding efficiency according to image characteristics, or if necessary, the coding unit may be recursively divided into coding units with a deeper depth and a coding unit with an optimal size may be used as the final coding unit. Here, the encoding process may include a process of prediction, transformation, and reconstruction (to be described later). As another example, the processor may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be split or divided from the above-mentioned final coding unit. The prediction unit may be a unit for sample prediction, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient.

[0085] In some cases, a unit may be used interchangeably with terms such as a block or region. In general, an M×N block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component. A sample may be used as a term corresponding to one picture (or image) of a pixel or picture element.

[0086] In the encoding device 200, a prediction signal (prediction block, prediction sample array) output from the inter predictor 221 or the intra predictor 222 is subtracted from an input image signal (original block, original sample array) to generate a residual signal (residual block, residual sample array), and the generated residual signal is sent to the transformer 232. In this case, as shown, a unit that subtracts a prediction signal (prediction block, prediction sample array) from an input image signal (original block, original sample array) in the encoding device 200 may be referred to as a subtractor 231. The predictor may perform prediction on a block to be processed (hereinafter, referred to as a current block) and generate a prediction block including prediction samples of the current block. The predictor may determine whether intra prediction or inter prediction is applied based on the current block or CU. As described later in the description of each prediction mode, the predictor may generate various information related to prediction (e.g., prediction mode information) and send the generated information to the entropy encoder 240. Information about the prediction may be encoded in the entropy encoder 240 and output in the form of a bitstream.

[0087] The intra-frame predictor 222 may predict the current block with reference to samples in the current picture. Depending on the prediction mode, the referenced samples may be located near the current block or may be spaced apart. In intra-frame prediction, the prediction mode may include multiple non-directional modes and multiple directional modes. For example, the non-directional mode may include a DC mode and a plane mode. For example, depending on the level of detail of the prediction direction, the directional mode may include 33 directional prediction modes or 65 directional prediction modes. However, this is only an example, and more or fewer directional prediction modes may be used depending on the settings. The intra-frame predictor 222 may use the prediction mode applied to the neighboring blocks to determine the prediction mode applied to the current block.

[0088] The inter-frame predictor 221 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. Here, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter-frame predictor 221 may configure a motion information candidate list based on the neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter-frame prediction may be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter-frame predictor 221 may use the motion information of the neighboring block as the motion information of the current block. In the skip mode, unlike the merge mode, the residual signal may not be transmitted. In the case of the motion vector prediction (MVP) mode, the motion vector of the neighboring block may be used as a motion vector predictor, and the motion vector of the current block may be indicated by signaling the motion vector difference.

[0089] The predictor 220 may generate a prediction signal based on various prediction methods described below. For example, the predictor may not only apply intra prediction or inter prediction to predict a block, but also apply both intra prediction and inter prediction at the same time. This may be referred to as combined inter and intra prediction (CIIP). In addition, the predictor may predict a block based on an intra block copy (IBC) prediction mode or a palette mode. The IBC prediction mode or the palette mode may be used for content image / video encoding of games, etc., such as screen content coding (SCC). IBC basically performs prediction in the current picture, but may be performed similarly to inter prediction so that a reference block is derived in the current picture. That is, IBC may use at least one inter prediction technique described in the present disclosure. The palette mode may be considered as an example of intra coding or intra prediction. When the palette mode is applied, the sample values ​​within the picture may be signaled based on information about the palette table and the palette index.

[0090] The prediction signal generated by the predictor (including the inter-frame predictor 221 and / or the intra-frame predictor 222) can be used to generate a reconstruction signal or to generate a residual signal. The transformer 232 can generate a transform coefficient by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a karhunen-loève transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, when the relationship information between pixels is represented by a graph, GBT means a transform obtained from a graph. CNT refers to a transform generated based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process can be applied to square pixel blocks of the same size or to blocks of variable size other than squares.

[0091] The quantizer 233 may quantize the transform coefficients and send them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information about the quantized transform coefficients) and output a bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantizer 233 may rearrange the block type quantized transform coefficients into a one-dimensional vector form based on the coefficient scanning order, and generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form. Information about the transform coefficients may be generated. The entropy encoder 240 may perform various encoding methods such as exponential Golomb, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), etc. The entropy encoder 240 may encode information required for video / image reconstruction other than the quantized transform coefficients (e.g., the value of a syntax element, etc.) together or separately. The encoded information (e.g., encoded video / image information) may be transmitted or stored in units of NAL (Network Abstraction Layer) in the form of a bitstream. The video / image information may also include information about various parameter sets, such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. In the present disclosure, information and / or syntax elements sent / signaled to a decoding device from an encoding device may be included in the video / image information. The video / image information may be encoded by the above-mentioned encoding process and included in a bitstream. The bitstream may be sent via a network or may be stored in a digital storage medium. The network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) that sends a signal output from the entropy encoder 240 and / or a storage unit (not shown) that stores the signal may be included as an internal / external element of the encoding device 200, and alternatively, the transmitter may be included in the entropy encoder 240.

[0092] The quantized transform coefficients output from the quantizer 233 can be used to generate a prediction signal. For example, a residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transformation to the quantized transform coefficients via the dequantizer 234 and the inverse transformer 235. The adder 250 adds the reconstructed residual signal to the prediction signal output from the inter-frame predictor 221 or the intra-frame predictor 222 to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). If there is no residual in the block to be processed (for example, when the skip mode is applied), the prediction block can be used as a reconstructed block. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. As described below, the generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture and can be used for inter-frame prediction of the next picture by filtering.

[0093] Furthermore, luma mapping with chroma scaling (LMCS) may be applied during picture encoding and / or reconstruction.

[0094] The filter 260 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 260 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and store the modified reconstructed picture in the memory 270 (specifically, the DPB of the memory 270). For example, various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filter 260 can generate various information related to filtering and send the generated information to the entropy encoder 240, as described later in the description of each filtering method. The information related to filtering can be encoded by the entropy encoder 240 and output in the form of a bit stream.

[0095] The modified reconstructed picture transmitted to the memory 270 may be used as a reference picture in the inter predictor 221. When inter prediction is applied by the encoding apparatus, prediction mismatch between the encoding apparatus 200 and the decoding apparatus 300 may be avoided and encoding efficiency may be improved.

[0096] The DPB of the memory 270 may store a modified reconstructed picture used as a reference picture in the inter-frame predictor 221. The memory 270 may store motion information of a block from which motion information in the current picture is derived (or encoded) and / or motion information of a reconstructed block in the picture. The stored motion information may be sent to the inter-frame predictor 221 and used as motion information of a spatially adjacent block or motion information of a temporally adjacent block. The memory 270 may store reconstructed samples of a reconstructed block in the current picture and may transmit the reconstructed samples to the intra-frame predictor 222.

[0097] Figure 3 is a schematic diagram showing the configuration of a video / image decoding device to which an embodiment of the present disclosure can be applied.

[0098] Reference Figure 3 , the decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 332 and an intra-frame predictor 331. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322. According to an embodiment, the entropy decoder 310, the residual processor 320, the predictor 330, the adder 340, and the filter 350 may be configured by a hardware component (e.g., a decoder chipset or a processor). In addition, the memory 360 may include a decoded picture buffer (DPB) or may be configured by a digital storage medium. The hardware component may also include a memory 360 as an internal / external component.

[0099] When a bit stream including video / image information is input, the decoding device 300 can reconstruct the bit stream corresponding to the bit stream in the video / image format. Figure 2 The image corresponding to the processing of the video / image information in the encoding device. For example, the decoding device 300 may derive the unit / block based on the block segmentation related information obtained from the bit stream. The decoding device 300 may use a processor applied in the encoding device to perform decoding. Therefore, for example, the decoding processor may be a coding unit, and the coding unit may be divided from a coding tree unit or a maximum coding unit according to a quadtree structure, a binary tree structure and / or a ternary tree structure. One or more transform units may be derived from the coding unit. The reconstructed image signal decoded and output by the decoding device 300 may be reproduced by a reproduction device.

[0100] The decoding device 300 may receive Figure 2The received signal may be decoded by the entropy decoder 310. For example, the entropy decoder 310 may parse the bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. The decoding device may also decode the picture based on the information about the parameter set and / or the general constraint information. The information and / or syntax elements signaled / received described later in this disclosure may be decoded and obtained from the bitstream by a decoding process. For example, the entropy decoder 310 may decode the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients of the residual. More specifically, the CABAC entropy decoding method may receive bins corresponding to each syntax element in the bitstream, determine a context model using information of a decoding target syntax element, decoding information of a decoding target block, or information of a symbol / bin decoded in a previous stage, and perform arithmetic decoding on the bin by predicting the probability of occurrence of the bin according to the determined context model, and generate symbols corresponding to the values ​​of each syntax element. In this case, the CABAC entropy decoding method may update the context model by using the information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model. Information related to prediction among the information decoded by the entropy decoder 310 may be provided to the predictor (inter-frame predictor 332 and intra-frame predictor 331), and the residual value (i.e., quantized transform coefficients and related parameter information) for which entropy decoding is performed in the entropy decoder 310 may be input to the residual processor 320. The residual processor 320 may derive a residual signal (residual block, residual sample, residual sample array). In addition, information about filtering among the information decoded by the entropy decoder 310 may be provided to the filter 350. In addition, a receiver (not shown) for receiving a signal output from the encoding device may also be configured as an internal / external element of the decoding device 300, or the receiver may be a component of the entropy decoder 310. In addition, the decoding device according to the present disclosure may be referred to as a video / image / picture decoding device, and the decoding device may be classified into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoder 310, and the sample decoder may include at least one of the dequantizer 321, the inverse transformer 322, the adder 340, the filter 350, the memory 360, the inter-frame predictor 332, and the intra-frame predictor 331.

[0101] The dequantizer 321 may dequantize the quantized transform coefficient and output the transform coefficient. The dequantizer 321 may rearrange the quantized transform coefficient in a two-dimensional block form. In this case, the rearrangement may be performed based on a coefficient scanning order performed in the encoding device. The dequantizer 321 may perform dequantization on the quantized transform coefficient using a quantization parameter (e.g., quantization step size information) and obtain the transform coefficient.

[0102] The inverse transformer 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0103] The predictor may perform prediction on the current block and generate a prediction block including prediction samples of the current block. The predictor may determine whether to apply intra prediction or inter prediction to the current block based on information on prediction output from the entropy decoder 310 and may determine a specific intra / inter prediction mode.

[0104] The predictor 330 may generate a prediction signal based on various prediction methods described below. For example, the predictor may not only apply intra prediction or inter prediction to predict a block, but may also apply intra prediction and inter prediction at the same time. This may be referred to as combined inter and intra prediction (CIIP). In addition, the predictor may predict a block based on an intra block copy (IBC) prediction mode or a palette mode. The IBC prediction mode or the palette mode may be used for content image / video encoding of games, etc., such as screen content coding (SCC). IBC basically performs a prediction in the current picture, but may be performed similarly to inter prediction so that a reference block is derived in the current picture. That is, IBC may use at least one inter prediction technique described in the present disclosure. The palette mode may be considered as an example of intra coding or intra prediction. When the palette mode is applied, the sample values ​​within the picture may be signaled based on information about the palette table and the palette index.

[0105] The intra-frame predictor 331 may predict the current block with reference to samples in the current picture. Depending on the prediction mode, the referenced samples may be located near the current block or may be spaced apart. In intra-frame prediction, the prediction mode may include a plurality of non-directional modes and a plurality of directional modes. The intra-frame predictor 331 may determine the prediction mode applied to the current block using the prediction mode applied to the neighboring blocks.

[0106] The inter-frame predictor 332 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter-frame predictor 332 may configure a motion information candidate list based on neighboring blocks and derive a motion vector and / or a reference picture index of the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes, and information about the prediction may include information indicating an inter-frame prediction mode of the current block.

[0107] The adder 340 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including the inter-frame predictor 332 and / or the intra-frame predictor 331). If there is no residual in the block to be processed, for example, when the skip mode is applied, the prediction block can be used as the reconstructed block.

[0108] The adder 340 may be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next block to be processed in the current picture, may be output through filtering as described below, or may be used for inter prediction of the next picture.

[0109] In addition, luma mapping with chroma scaling (LMCS) may be applied in the picture decoding process.

[0110] The filter 350 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 350 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and store the modified reconstructed picture in the memory 360 (specifically, the DPB of the memory 360). For example, the various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0111] The (modified) reconstructed picture stored in the DPB of the memory 360 may be used as a reference picture in the inter-frame predictor 332. The memory 360 may store the motion information of the block from which the motion information in the current picture is derived (or decoded) and / or the motion information of the reconstructed block in the picture. The stored motion information may be sent to the inter-frame predictor 221 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 360 may store the reconstructed samples of the reconstructed block in the current picture and transmit the reconstructed samples to the intra-frame predictor 331.

[0112] In this document, the embodiments described in the filter 260, the inter-frame predictor 221, and the intra-frame predictor 222 of the encoding device 200 may be the same as or respectively applied to the filter 350, the inter-frame predictor 332, and the intra-frame predictor 331 of the decoding device 300. This may also be applied to the inter-frame predictor 332 and the intra-frame predictor 331.

[0113] As described above, in video encoding, prediction is performed to increase compression efficiency. Thus, a prediction block including prediction samples of a current block (block to be encoded) can be generated. Here, the prediction block includes prediction samples in a spatial domain (or pixel domain). The prediction block is derived identically from an encoding device and a decoding device, and the encoding device decodes information about the residual between the original block and the prediction block (residual information) rather than the original sample value of the original block itself. By notifying the device with a signal, image coding efficiency can be increased. The decoding device can derive a residual block including residual samples based on the residual information, and generate a reconstructed block including reconstructed samples by summing the residual block and the prediction block, and generate a reconstructed picture including the reconstructed block.

[0114] The residual information may be generated by a transform process and a quantization process. For example, the encoding device may derive a residual block between the original block and the prediction block, and perform a transform process on the residual samples (residual sample array) included in the residual block to derive a transform coefficient, and then, by performing a quantization process on the transform coefficient, derive the quantized transform coefficient to signal the residual related information (via a bitstream) to the decoding device. Here, the residual information may include position information, transform technology, transform kernel and quantization parameter, value information of the quantized transform coefficient, etc. The decoding device may perform a dequantization / inverse transform process based on the residual information and derive residual samples (or residual blocks). The decoding device may generate a reconstructed picture based on the prediction block and the residual block. The encoding device may also dequantize / inverse transform the quantized transform coefficients for reference to inter-frame prediction of a later picture to derive a residual block, and generate a reconstructed picture based thereon.

[0115] In this document, at least one of quantization / dequantization and / or transform / inverse transform may be omitted. When quantization / dequantization is omitted, the quantized transform coefficient may be referred to as a transform coefficient. When transform / inverse transform is omitted, the transform coefficient may be referred to as a coefficient or a residual coefficient, or may still be referred to as a transform coefficient for consistency of expression.

[0116] In this document, the quantized transform coefficient and the transform coefficient may be referred to as a transform coefficient and a scaled transform coefficient, respectively. In this case, the residual information may include information about the transform coefficient, and the information about the transform coefficient may be signaled by a residual coding syntax. The transform coefficient may be derived based on the residual information (or information about the transform coefficient), and the scaled transform coefficient may be derived by inverse transforming (scaling) the transform coefficient. The residual sample may be derived based on an inverse transform (transform) of the scaled transform coefficient. This may also be applied / expressed in other parts of this document.

[0117] Intra-frame prediction may refer to the prediction of the prediction sample of the current block generated based on the reference sample in the picture to which the current block belongs (hereinafter referred to as the current picture). When intra-frame prediction is applied to the current block, the neighboring reference samples to be used for intra-frame prediction of the current block may be derived. The neighboring reference samples of the current block may include samples adjacent to the left boundary of the current block of size nW×nH and a total of 2×nH samples adjacent to the lower left, samples adjacent to the upper boundary of the current block and a total of 2×nW samples adjacent to the upper right, and one sample adjacent to the upper left of the current block. Alternatively, the neighboring reference samples of the current block may include multiple upper neighboring samples and multiple left neighboring samples. In addition, the neighboring reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block of size nW×nH, a total of nW samples adjacent to the lower boundary of the current block, and one sample adjacent to the lower right of the current block.

[0118] However, some neighboring reference samples of the current block may not be decoded or available. In this case, the decoder can configure the neighboring reference samples to be used for prediction by replacing the unavailable samples with available samples. Alternatively, the neighboring reference samples to be used for prediction can be configured by interpolation of available samples.

[0119] When deriving neighboring reference samples, (i) the prediction sample may be derived based on an average or interpolation of neighboring reference samples of the current block, and (ii) the prediction sample may be derived based on a reference sample existing in a specific (prediction) direction of the prediction sample among peripheral reference samples of the current block. The case of (i) may be referred to as a non-directional mode or a non-angular mode, and the case of (ii) may be referred to as a directional mode or an angular mode.

[0120] In addition, the prediction sample may also be generated by interpolation between a second neighboring sample and a first neighboring sample located in a direction opposite to the prediction direction of the intra prediction mode of the current block based on the prediction sample of the current block among the neighboring reference samples. The above situation may be referred to as linear interpolation intra prediction (LIP). In addition, a linear model may be used to generate a chrominance prediction sample based on a luma sample. This situation may be referred to as an LM mode.

[0121] In addition, the temporary prediction sample of the current block can be derived based on the filtered neighboring reference sample, and at least one reference sample derived according to the intra prediction mode among the existing neighboring reference samples (i.e., the unfiltered neighboring reference sample) and the temporary prediction sample can be weighted and summed to derive the prediction sample of the current block. The above situation can be called position-dependent intra prediction (PDPC).

[0122] In addition, the reference sample row with the highest prediction accuracy among the neighboring multiple reference sample rows of the current block can be selected to derive the prediction sample using the reference sample located in the prediction direction on the corresponding row, and then the reference sample row used in this article can be indicated (notified by signal) to the decoding device, thereby performing intra-frame prediction encoding. The above situation can be called multi-reference row (MRL) intra-frame prediction or MRL-based intra-frame prediction.

[0123] In addition, intra prediction can be performed by dividing the current block into vertical or horizontal sub-partitions based on the same intra prediction mode, and the neighboring reference samples can be derived and used in units of sub-partitions. That is, in this case, the intra prediction mode of the current block is also applicable to the sub-partitions, and in some cases, the intra prediction performance can be improved by deriving and using the neighboring reference samples in units of sub-partitions. This prediction method may be referred to as intra-sub-partition (ISP) or ISP-based intra prediction.

[0124] The above-mentioned intra-frame prediction method may be separately referred to as an intra-frame prediction type from the intra-frame prediction mode. The intra-frame prediction type may be referred to by various terms such as an intra-frame prediction technique or an additional intra-frame prediction mode. For example, the intra-frame prediction type (or additional intra-frame prediction mode) may include at least one of the above-mentioned LIP, PDPC, MRL, and ISP. General intra-frame prediction methods other than specific intra-frame prediction types such as LIP, PDPC, MRL, or ISP may be referred to as normal intra-frame prediction types. The normal intra-frame prediction type may generally be applied when a specific intra-frame prediction type is not applied, and prediction may be performed based on the above-mentioned intra-frame prediction mode. In addition, post-filtering may be performed on the derived prediction samples as needed.

[0125] Specifically, the intra prediction process may include an intra prediction mode / type determination step, a neighboring reference sample derivation step, and a prediction sample derivation step based on the intra prediction mode / type. In addition, a post-filtering step may be performed on the derived prediction samples as needed.

[0126] When intra prediction is applied, the intra prediction mode applied to the current block may be determined using the intra prediction mode of the neighboring blocks. For example, the decoding device may select one of the mpm candidates of the mpm list derived from the intra prediction mode of the neighboring blocks (e.g., left and / or upper neighboring blocks) of the current block based on the received most probable mode (mpm) index, and select one of the other remaining intra prediction modes not included in the mpm candidates (and the plane mode) based on the remaining intra prediction mode information. The mpm list may be configured to include or exclude the plane mode as a candidate. For example, if the mpm list includes the plane mode as a candidate, the mpm list may have six candidates. If the mpm list does not include the plane mode as a candidate, the mpm list may have three candidates. When the mpm list does not include the plane mode as a candidate, a non-planar flag (e.g., intra_luma_not_planar_flag) indicating whether the intra prediction mode of the current block is not a plane mode may be signaled. For example, the mpm flag may be first signaled, and when the value of the mpm flag is 1, the mpm index and the non-planar flag may be signaled. In addition, the mpm index may be signaled when the value of the non-planar flag is 1. Here, the mpm list is configured to not include the planar mode as a candidate without first signaling the non-planar flag to first check whether it is the planar mode, because the planar mode is always regarded as the mpm.

[0127] For example, whether the intra prediction mode applied to the current block is in the mpm candidate (and planar mode) or in the remaining mode may be indicated based on an mpm flag (e.g., Intra_luma_mpm_flag). A value of 1 for the mpm flag may indicate that the intra prediction mode of the current block is within the mpm candidate (and planar mode), and a value of 0 for the mpm flag may indicate that the intra prediction mode of the current block is not within the mpm candidate (and planar mode). A value of 0 for a non-planar flag (e.g., Intra_luma_not_planar_flag) may indicate that the intra prediction mode of the current block is a planar mode, and a value of 1 for the non-planar flag value may indicate that the intra prediction mode of the current block is not a planar mode. The mpm index may be signaled in the form of an mpm_idx or intra_luma_mpm_idx syntax element, and the remaining intra prediction mode information may be signaled in the form of a rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax element. For example, the remaining intra prediction mode information may index the remaining intra prediction modes not included in the mpm candidates (and planar modes) among all intra prediction modes in the order of the prediction mode numbers to indicate one of them. The intra prediction mode may be an intra prediction mode of a luminance component (sample). Below, the intra prediction mode information may include at least one of an mpm flag (e.g., Intra_luma_mpm_flag), a non-planar flag (e.g., Intra_luma_not_planar_flag), an mpm index (e.g., mpm_idx or intra_luma_mpm_idx), and remaining intra prediction mode information (rem_intra_luma_pred_mode or intra_luma_mpm_remainder). In this document, the MPM list may be referred to as various terms such as MPM candidate list and candModeList. When MIP is applied to the current block, a separate mpm flag (eg, intra_mip_mpm_flag), an mpm index (eg, intra_mip_mpm_idx), and remaining intra prediction mode information (eg, intra_mip_mpm_remainder) for the MIP may be signaled, and a non-planar flag may not be signaled.

[0128] In other words, generally, when block splitting is performed on an image, the current block to be encoded and the neighboring blocks have similar image characteristics. Therefore, the probability that the current block and the neighboring blocks have the same or similar intra-frame prediction mode is high. Therefore, the encoder can use the intra-frame prediction mode of the neighboring block to encode the intra-frame prediction mode of the current block.

[0129] For example, the encoder / decoder may configure a list of most probable modes (MPMs) for the current block. The MPM list may also be referred to as an MPM candidate list. Herein, MPM may refer to a mode that improves coding efficiency by considering the similarity between the current block and the neighboring blocks in intra-frame prediction mode encoding. As described above, the MPM list may be configured to include a planar mode, or may be configured not to include a planar mode. For example, when the MPM list includes a planar mode, the number of candidates in the MPM list may be 6. And, if the MPM list does not include a planar mode, the number of candidates in the MPM list may be 5.

[0130] The encoder / decoder can be configured with an MPM list consisting of 5 or 6 MPMs.

[0131] To configure the MPM list, three types of modes may be considered: default intra mode, neighbor intra mode, and derived intra mode.

[0132] For the neighboring intra mode, two neighboring blocks may be considered, ie, a left neighboring block and an above neighboring block.

[0133] As described above, if the MPM list is configured not to include the planar mode, the planar mode is excluded from the list and the number of MPM list candidates may be set to five.

[0134] In addition, the non-directional mode (or non-angular mode) among the intra prediction modes may include a DC mode based on an average of neighboring reference samples of the current block or a planar mode based on interpolation.

[0135] When inter-frame prediction is applied, the predictor of the encoding device / decoding device can derive prediction samples by performing inter-frame prediction in units of blocks. Inter-frame prediction can be a prediction derived in a manner that depends on data elements (e.g., sample values ​​or motion information) of a picture other than the current picture. When inter-frame prediction is applied to the current block, the prediction block (prediction sample array) of the current block can be derived based on the reference block (reference sample array) specified by the motion vector on the reference picture indicated by the reference picture index. Here, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information of the current block can be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction type (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. A temporally neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc., and a reference picture including a temporally neighboring block may be referred to as a collocated picture (colPic). For example, a motion information candidate list may be configured based on a neighboring block of a current block, and a flag or index information indicating which candidate is selected (used) to derive a motion vector and / or a reference picture index of the current block may be signaled. Inter-frame prediction may be performed based on various prediction modes. For example, in the case of a skip mode and a merge mode, the motion information of the current block may be the same as the motion information of the neighboring block. In the skip mode, unlike the merge mode, a residual signal may not be sent. In the case of a motion vector prediction (MVP) mode, the motion vector of the selected neighboring block may be used as a motion vector predictor, and the motion vector of the current block may be signaled. In this case, the motion vector of the current block may be derived using the sum of the motion vector predictor and the motion vector difference.

[0136] According to the inter prediction type (L0 prediction, L1 prediction, Bi prediction, etc.), the motion information may include L0 motion information and / or L1 motion information. The motion vector in the L0 direction may be referred to as the L0 motion vector or MVL0, and the motion vector in the L1 direction may be referred to as the L1 motion vector or MVL1. Prediction based on the L0 motion vector may be referred to as the L0 prediction, prediction based on the L1 motion vector may be referred to as the L1 prediction, and prediction based on both the L0 motion vector and the L1 motion vector may be referred to as the dual prediction. Here, the L0 motion vector may indicate a motion vector associated with the reference picture list L0 (L0), and the L1 motion vector may indicate a motion vector associated with the reference picture list L1 (L1). The reference picture list L0 may include a picture earlier than the current picture in output order as a reference picture, and the reference picture list L1 may include a picture later than the current picture in output order. The previous picture may be referred to as a forward (reference) picture, and the subsequent picture may be referred to as a backward (reference) picture. The reference picture list L0 may also include a picture later than the current picture in output order as a reference picture. In this case, the previous picture may be indexed first in the reference picture list L0, and the subsequent picture may be indexed later. The reference picture list L1 may also include a previous picture earlier than the current picture in the output order as a reference picture. In this case, the subsequent picture may be indexed first in the reference picture list 1, and the previous picture may be indexed later. The output order may correspond to a picture order count (POC) order.

[0137] Figure 4 An exemplary block tree structure is shown. Figure 4 It is exemplarily shown that a CTU is divided into a plurality of CUs based on a quadtree and a nested multi-type tree structure.

[0138] Bold block edges represent quadtree partitioning, and the remaining edges represent multi-type tree partitioning. Quadtree partitioning with nested multi-type trees can provide a content-adaptive coding tree structure. A CU may correspond to a coding block (CB). Alternatively, a CU may include a coding block of luma samples and two coding blocks of corresponding chroma samples. The size of a CU may be as large as a CTU, or as small as 4×4 of a luma sample unit. For example, in the case of a 4:2:0 color format (or chroma format), the maximum chroma CB size may be 64×64, and the minimum chroma CB size may be 2×2.

[0139] In this document, for example, the maximum allowed luma TB size may be 64 × 64, and the maximum allowed chroma TB size may be 32 × 32. When the width or height of the CB divided according to the tree structure is greater than the maximum transform width or the maximum transform height, the corresponding CB may be automatically (or implicitly) divided until the TB size limits in the horizontal and vertical directions are met.

[0140] In this document, the coding tree scheme may support that the luminance (component) block and the chrominance (component) block have separate block tree structures. Blocks with separate block tree structures may be blocks that have been coded as separate trees. The case where the luminance block and the chrominance block in one CTU have the same block tree structure may be represented as SINGLE_TREE (single tree structure). The case where the luminance block and the chrominance block in one CTU have separate block tree structures may be represented as DUAL_TREE (dual tree structure). In this case, the block tree type of the luminance component may be referred to as DUAL_TREE_LUMA, and the block tree type of the chrominance component may be referred to as DUAL_TREE_CHROMA. Blocks with a dual tree structure may be blocks that have been coded as dual trees. For P and B slices / block groups, the luminance CTB and chrominance CTB in one CTU may be restricted to have the same coding tree structure. However, for I slices / block groups, luminance blocks and chrominance blocks may have separate block tree structures, respectively. When a separate block tree mode is applied, luma CTBs may be divided into CUs based on a specific coding tree structure, and chroma CTBs may be divided into chroma CUs based on another coding tree structure. This may mean that a CU in an I slice / patch group consists of coding blocks of a luma component or coding blocks of two chroma components, and a CU in a P or B slice / patch group consists of blocks of three color components. In this document, a slice may be referred to as a patch / patch group, and a patch / patch group may be referred to as a slice.

[0141] Although a quadtree coding tree structure with a multi-type tree has been described, the structure of dividing the CU is not limited thereto. For example, the BT structure and the TT structure may be interpreted as concepts included in a multi-partition tree (MPT) structure, and the CU may be interpreted as being divided by the QT structure and the MPT structure. In an example in which the CU is divided by the QT structure and the MPT structure, the partition structure may be determined by signaling a syntax element (e.g., MPT_split_type) including information about how many blocks a leaf node of the QT structure is divided into and a syntax element (e.g., MPT_split_mode) including information about whether the leaf node of the QT structure is divided in a vertical direction or a horizontal direction.

[0142] In addition, in another example, the CU may be divided using any other method other than the QT structure, the BT structure, or the TT structure. That is, unlike dividing a CU at a lower depth into CUs of 1 / 4 size at a higher depth according to the QT structure, or dividing a CU at a lower depth into CUs of 1 / 2 size at a higher depth according to the BT structure, or dividing a CU at a lower depth into CUs of 1 / 4 or 1 / 2 size at a higher depth according to the TT structure, a CU at a lower depth may be divided into CUs of 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 size at a higher depth according to circumstances, and the CU partitioning method is not limited thereto.

[0143] Figure 5 The hierarchical structure of the encoded image / video is shown exemplarily.

[0144] Reference Figure 5 , the encoded image / video is divided into a video coding layer (VCL) that handles the image / video and its own decoding processing, a subsystem that sends and stores the encoded information, and a NAL (Network Abstraction Layer) that is responsible for functions and exists between the VCL and the subsystem.

[0145] In the VCL, VCL data including compressed image data (slice data) is generated, or a parameter set including a picture parameter set (PSP), a sequence parameter set (SPS) and a video parameter set (VPS) or a supplemental enhancement information (SEI) message additionally required for image decoding processing may be generated.

[0146] In NAL, a NAL unit may be generated by adding header information (NAL unit header) to a raw byte sequence payload (RBSP) generated in a VCL. In this case, the RBSP refers to slice data, parameter sets, SEI messages, etc. generated in the VCL. The NAL unit header may include NAL unit type information specified according to the RBSP data included in the corresponding NAL unit.

[0147] As shown in the figure, NAL units can be classified into VCL NAL units and non-VCL NAL units according to RBSP generated in VCL. VCL NAL units may refer to NAL units including information about images (slice data), and non-VCL NAL units may refer to NAL units including information required for decoding images (parameter sets or SEI messages).

[0148] The above-mentioned VCL NAL unit and non-VCL NAL unit can be sent through the network by adding header information according to the data standard of the subsystem. For example, the NAL unit can be converted into a data format of a predetermined standard such as H.266 / VVC file format, real-time transport protocol (RTP), transport stream (TS), etc. and sent through various networks.

[0149] As described above, the NAL unit may be specified with a NAL unit type according to the RBSP data structure included in the corresponding NAL unit, and information on the NAL unit type may be stored in a NAL unit header and signaled.

[0150] For example, NAL units may be classified into VCL NAL unit types and non-VCL NAL unit types according to whether the NAL unit includes information about an image (slice data). VCL NAL unit types may be classified according to the nature and type of a picture included in a VCL NAL unit, and non-VCL NAL unit types may be classified according to the type of a parameter set.

[0151] The following are examples of NAL unit types specified according to the type of parameter sets included in a non-VCL NAL unit type.

[0152] -APS (Adaptation Parameter Set) NAL unit: the type of NAL unit that includes APS

[0153] -DPS (Decoding Parameter Set) NAL unit: the type of NAL unit that includes the DPS

[0154] -VPS (Video Parameter Set) NAL unit: the type of NAL unit that contains the VPS

[0155] -SPS (Sequence Parameter Set) NAL unit: the type of NAL unit that includes SPS

[0156] -PPS (Picture Parameter Set) NAL unit: the type of NAL unit that includes the PPS

[0157] - PH (Picture Header) NAL unit: the type of NAL unit that includes PH

[0158] The above-mentioned NAL unit type may have syntax information of the NAL unit type, and the syntax information may be stored in the NAL unit header and notified by a signal. For example, the syntax information may be nal_unit_type, and the NAL unit type may be specified by the nal_unit_type value.

[0159] In addition, as described above, a picture may include multiple slices, and a slice may include a slice header and slice data. In this case, a picture header may be further added to the multiple slices (slice header and slice data set) in a picture. The picture header (picture header syntax) may include information / parameters generally applicable to a picture. In this document, slices may be mixed with or replaced by tiling groups. In addition, in this document, slice headers may be mixed with or replaced by tiling group headers.

[0160] The slice header (slice header syntax) may include information / parameters that are generally applicable to slices. APS (APS syntax) or PPS (PPS syntax) may include information / parameters that are generally applicable to one or more slices or pictures. SPS (SPS syntax) may include information / parameters that are generally applicable to one or more sequences. VPS (VPS syntax) may include information / parameters that are generally applicable to multiple layers. DPS (DPS syntax) may include information / parameters that are generally applicable to the total video. DPS may include information / parameters related to the concatenation of coded video sequences (CVS). The high-level syntax (HLS) in this document may include at least one of APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, and slice header syntax.

[0161] In this document, the image / image information encoded from the encoding device and notified to the decoding device in the form of a bit stream by a signal includes not only the segmentation-related information, intra-frame / inter-frame prediction information, residual information, loop filtering information, etc. in the picture, but also includes the information included in the slice header, the information included in the APS, the information included in the PPS, the information included in the SPS and / or the information included in the VPS.

[0162] In addition, in order to compensate for the difference between the original image and the reconstructed image due to errors occurring in the compression encoding process such as quantization, loop filtering processing may be performed on the reconstructed sample or the reconstructed picture as described above. As described above, loop filtering may be performed by a filter of an encoding device and a filter of a decoding device, and a deblocking filter, SAO and / or an adaptive loop filter (ALF) may be applied. For example, the ALF process may be performed after the deblocking filter process and / or the SAO process are completed. However, even in this case, the deblocking filter process and / or the SAO process may be omitted.

[0163] Furthermore, to increase coding efficiency, Luma Mapping and Chroma Scaling (LMCS) may be applied as described above. LMCS may be referred to as a loop shaper (shaping). To increase coding efficiency, signaling of LMCS control and / or LMCS-related information may be performed hierarchically.

[0164] Figure 6 The hierarchical structure of the CVS according to the embodiment of this document is exemplarily shown.

[0165] Reference Figure 6 , a coded video sequence (CVS) may include an SPS, one or more PPSs (picture parameter sets), and one or more subsequent coded pictures. Each coded picture may be divided into rectangular areas. The rectangular areas may be referred to as tiles. The collected one or more tiles may form a tile group or a slice. In this case, the tile group header may be linked to a picture parameter set (PPS), and the PPS may be linked to the SPS.

[0166] Figure 7 The hierarchical structure of the CVS according to the embodiment of this document is exemplarily shown. Figure 8 The hierarchical structure of a CVS according to another embodiment of this document is exemplarily shown.

[0167] Reference Figure 8 , a coded video sequence (CVS) may include an SPS, a PPS, a patch group header, patch data and / or a CTU. Here, the patch group header and patch data may be referred to as a slice header and slice data, respectively.

[0168] The SPS may include a local flag to enable the tool to be used in the CVS. In addition, the SPS may be referenced by a PPS including information about parameters that change for each picture. Each coded picture may include one or more coded rectangular domain tiles. The tiles may be grouped into raster scans that form tile groups. Each tile group is encapsulated with header information called a tile group header. Each tile consists of a CTU that includes coded data. Here, the data may include original sample values, predicted sample values, and their luminance components and chrominance components (luminance predicted sample values ​​and chrominance predicted sample values).

[0169] According to the existing method, ALF data (ALF parameters) or LMCS data (LMCS parameters) is incorporated into the tile group header. Since one video is composed of multiple pictures and one picture includes multiple tiles, frequently signaling ALF data (ALF parameters) or LMCS data (LMCS parameters) in units of tile groups leads to a problem of reduced coding efficiency.

[0170] According to the embodiments proposed in this document, ALF parameters or LMCS data (LMCS parameters) may be incorporated into the APS and signaled as follows.

[0171] Reference Figure 7 In an embodiment, an APS is defined, and the APS may carry necessary ALF data (ALF parameters). In addition, the APS may have self-identification parameters, ALF data, and / or LMCS data. The self-identification parameters of the APS may include an APS ID. That is, the APS may include information indicating the APS ID. The patch group header or the slice header may refer to the APS using the APS index information. In other words, the patch group header or the slice header may include the APS index information, and the ALF process may be performed for the target block based on the LMCS data (LMCS parameters) included in the APS having the APS ID indicated by the APS index information, or the ALF process of the target block may be performed based on the ALF data (ALF parameters) included in the APS having the APS ID indicated by the APS index information. Here, the APS index information may be referred to as APS ID information.

[0172] In an example, the SPS may include a flag that allows the use of ALF. For example, when the CVS starts, the SPS may be checked, and the flag in the SPS may be checked. For example, the SPS may include the syntax of Table 1 below. The syntax in Table 1 may be part of the SPS.

[0173] [Table 1]

[0174]

[0175] For example, the semantics of the syntax elements included in the syntax of Table 1 may be represented as shown in the following table.

[0176] [Table 2]

[0177]

[0178]

[0179] That is, the sps_alf_enabled_flag syntax element may indicate whether ALF is enabled based on whether its value is 0 or 1. The sps_alf_enabled_flag syntax element may be referred to as an ALF enabled flag (this may be referred to as a first ALF enabled flag) and may be included in the SPS. That is, the ALF enabled flag may be signaled in the SPS (or SPS level). When the value of the ALF enabled flag signaled in the SPS is 1, the ALF may be determined to be substantially enabled for the picture of the reference SPS in the CVS. In addition, as described above, the ALF may be individually turned on / off by signaling an additional enabled flag at a lower level than the SPS.

[0180] For example, if the ALF tool is enabled for CVS, an additional enable flag (which may be referred to as a second ALF enable flag) may be signaled in a tile group header or a slice header. For example, when ALF is enabled at the SPS level, the second ALF enable flag may be parsed / signaled. When the value of the second ALF enable flag is 1, ALF data may be parsed through the tile group header or the slice header. For example, the second ALF enable flag may specify the ALF enable condition for the luma component and the chroma component. The ALF data may be accessed through the APS ID information.

[0181] [Table 3]

[0182]

[0183] [Table 4]

[0184]

[0185] For example, the semantics of the syntax elements included in the syntax of Table 3 or Table 4 above may be expressed as shown in the following table.

[0186] [Table 5]

[0187]

[0188] [Table 6]

[0189]

[0190] The second ALF enabled flag may include a tile_group_alf_enabled_flag syntax element or a slice_alf_enabled_flag syntax element.

[0191] The APS referenced by the corresponding tile group or the corresponding slice may be identified based on APS ID information (eg, tile_group_aps_id syntax element or slice_aps_id syntax element). The APS may include ALF data.

[0192] In addition, for example, the structure of the APS including the ALF data may be described based on the following syntax and semantics. The syntax of Table 7 may be a part of the APS.

[0193] [Table 7]

[0194]

[0195] [Table 8]

[0196]

[0197] As described above, the adaptation_parameter_set_id syntax element may indicate an identifier of the corresponding APS. That is, the APS may be identified based on the adaptation_parameter_set_id syntax element. The adaptation_parameter_set_id syntax element may be referred to as APS ID information. In addition, the APS may include an ALF data field. The ALF data field may be parsed / signaled after the adaptation_parameter_set_id syntax element.

[0198] Additionally, for example, an APS extension flag (e.g., an aps_extension_flag syntax element) may be parsed / signaled in the APS. The APS extension flag may indicate whether an APS extension data flag (aps_extension_data_flag) syntax element is present. For example, the APS extension flag may be used to provide an extension point for a later version of the VVC standard.

[0199] Fig. 9An exemplary LMCS structure according to an embodiment of this document is shown. Fig. 9 The LMCS structure 900 includes a loop mapping section 910 for luma components and a luma-dependent chroma residual scaling section 920 for chroma components based on an adaptive piecewise linear (adaptive PWL) model. The dequantization and inverse transform 911, reconstruction 912, and intra prediction 913 blocks of the loop mapping section 910 represent the processing applied in the mapped (shaped) domain. The loop filter 915, motion compensation or inter prediction 917 blocks of the loop mapping section 910, and the reconstruction 922, intra prediction 923, motion compensation or inter prediction 924, and loop filter 925 blocks of the chroma residual scaling section 920 represent the processing applied in the original (non-mapped, non-shaped) domain.

[0200] like Fig. 9 As shown, when LMCS is enabled, at least one of a reverse mapping (shaping) process 914, a forward mapping (shaping) process 918, and a chroma scaling process 921 may be applied. For example, a reverse mapping process may be applied to a (reconstructed) luma sample (or luma sample or luma sample array) in a reconstructed picture. The reverse mapping process may be performed based on a piecewise function (reverse) index of the luma sample. The piecewise function (reverse) index may identify the segment to which the luma sample belongs. The output of the reverse mapping process is a modified (reconstructed) luma sample (or modified luma sample or modified luma sample array). LMCS may be enabled or disabled at a tile group (or slice), picture, or higher level.

[0201] A forward mapping process and / or a chroma scaling process may be applied to generate a reconstructed picture. The picture may include luma samples and chroma samples. A reconstructed picture with luma samples may be referred to as a reconstructed luma picture, and a reconstructed picture with chroma samples may be referred to as a reconstructed chroma picture. A combination of a reconstructed luma picture and a reconstructed chroma picture may be referred to as a reconstructed picture. The reconstructed luma picture may be generated based on a forward mapping process. For example, if inter-frame prediction is applied to the current block, forward mapping is applied to luma prediction samples derived based on (reconstructed) luma samples in a reference picture. Since the (reconstructed) luma samples in the reference picture are generated based on a reverse mapping process, forward mapping may be applied to the luma prediction samples, and thus the mapped (shaped) luma prediction samples may be derived. The forward mapping process may be performed based on a piecewise function index of the luma prediction sample. The piecewise function index may be derived based on the value of the luma prediction sample or the value of the luma sample in the reference picture used for inter-frame prediction. If intra-frame prediction (or intra-block copy (IBC)) is applied to the current block, forward mapping is not required because the reverse mapping process has not been applied to the reconstructed samples in the current picture. The (reconstructed) luma samples in the reconstructed luma picture are generated based on the mapped luma prediction samples and the corresponding luma residual samples.

[0202] The reconstructed chroma picture may be generated based on a chroma scaling process. For example, the (reconstructed) chroma samples in the reconstructed chroma picture may be generated based on the chroma residual samples (c res ) and chroma prediction samples. Based on the (scaled) chroma residual samples (c resScale ) and the chroma residual scaling factor (cScaleInv can be called varScale) to derive the chroma residual samples (c res ). The chroma residual scaling factor may be calculated based on the shaped luma prediction sample value of the current block. For example, the chroma residual scaling factor may be calculated based on the shaped luma prediction sample value Y' pred The average brightness value ave(Y' pred ) to calculate the scaling factor. For reference, the (scaled) chroma residual samples derived based on inverse transform / dequantization may be referred to as c resScale , the chroma residual samples derived by performing the (inverse) scaling process on the (scaled) chroma residual samples may be referred to as c res .

[0203] Fig.10 A LMCS structure according to another embodiment of the present document is shown. Fig.10 Reference Fig. 9 Here, we mainly describe Fig.10 The LMCS structure and Fig. 9 The differences between the LMCS structures 900. Fig.10 The loop mapping part and the brightness-dependent chroma residual scaling part can be connected with Fig. 9 The loop mapping section 910 and the luma-dependent chroma residual scaling section 920 operate identically (similarly).

[0204] Reference Fig.10 , the chroma residual scaling factor can be derived based on the luma reconstruction samples. In this case, the average luma value (avgYr) can be obtained (derived) based on the neighboring luma reconstruction samples outside the reconstructed block, rather than the internal luma reconstruction samples of the reconstructed block, and the chroma residual scaling factor can be derived based on the average luma value (avgYr). Here, the neighboring luma reconstruction samples can be neighboring luma reconstruction samples of the current block, or can be neighboring luma reconstruction samples of a virtual pipeline data unit (VPDU) including the current block. For example, when intra prediction is applied to the target block, the reconstructed samples can be derived based on the prediction samples derived based on the intra prediction. In another example, when inter prediction is applied to the target block, forward mapping is applied to the prediction samples derived based on the inter prediction, and the reconstructed samples are generated (derived) based on the shaped (or forward mapped) luma prediction samples.

[0205] The video / image information notified by the bitstream signal may include LMCS parameters (information about LMCS). The LMCS parameters may be configured as high-level syntax (HLS, including slice header syntax), etc. A detailed description and configuration of the LMCS parameters will be described later. As described above, the syntax table described in this document (and the following embodiments) may be configured / encoded at the encoder end and notified to the decoder end via the bitstream signal. The decoder may parse / decode information about the LMCS in the syntax table (in the form of syntax components). One or more embodiments to be described below may be combined. The encoder may encode the current picture based on the information about the LMCS, and the decoder may decode the current picture based on the information about the LMCS.

[0206] The loop mapping of the luminance component can adjust the dynamic range of the input signal to improve compression efficiency by redistributing codewords across the dynamic range. For luminance mapping, a forward mapping (shaping) function (FwdMap) and an inverse mapping (shaping) function (InvMap) corresponding to the forward mapping function (FwdMap) can be used. The FwdMap function can be signaled using a piecewise linear model, for example, the piecewise linear model can have 16 segments or bins, and the segments can have equal lengths. In one example, the InvMap function does not need to be signaled, but is derived from the FwdMap function. That is, the inverse mapping can be a forward mapping function. For example, the inverse mapping function can be mathematically constructed as a symmetric function of the forward mapping, as reflected by the line y=x.

[0207] Loop (luminance) shaping can be used to map input luminance values ​​(samples) to changed values ​​in the shaped domain. The shaped values ​​can be encoded and then mapped back to the original (unmapped, unshaped) domain after reconstruction. To compensate for the interaction between the luminance signal and the chrominance signal, chrominance residual scaling can be applied. Loop shaping is accomplished by specifying a high-level syntax for a shaper model. The shaper model syntax can signal a piecewise linear model (PWL model). For example, the shaper model syntax can signal a PWL model with 16 bins or segments of the same length. A forward lookup table (FwdLUT) and / or an inverse lookup table (InvLUT) can be derived based on the piecewise linear model. For example, the PWL model pre-computes 1024-entry forward (FwdLUT) and inverse (InvLUT) lookup tables (LUTs). As an example, when the forward lookup table FwdLUT is derived, the inverse lookup table InvLUT can be derived based on the forward lookup table FwdLUT. The forward lookup table FwdLUT may map the input luminance value Yi to the modified value Yr, and the reverse lookup table InvLUT may map the modified value Yr to the reconstructed value Y′i. The reconstructed value Y′i may be derived based on the input luminance value Yi.

[0208] In one example, the SPS may include the syntax of Table 9 below. The syntax of Table 9 may include sps_reshaper_enabled_flag as a tool enable flag. Here, sps_reshaper_enabled_flag may be used to specify whether a reshaper is used in a coded video sequence (CVS). That is, sps_reshaper_enabled_flag may be a flag to enable reshaping in the SPS. In one example, the syntax of Table 9 may be part of the SPS.

[0209] [Table 9]

[0210]

[0211] In one example, the semantics of the syntax elements sps_seq_parameter_set_id and sps_reshaper_enabled_flag may be as shown in Table 10 below.

[0212] [Table 10]

[0213]

[0214] In one example, the tile group header or the slice header may include the syntax of Table 11 or Table 12 below.

[0215] [Table 11]

[0216]

[0217] [Table 12]

[0218]

[0219] The semantics of the syntax elements included in the syntax of Table 11 or Table 12 may include, for example, matters disclosed in the following table.

[0220] [Table 13]

[0221]

[0222] [Table 14]

[0223]

[0224]

[0225] As an example, once the reshaping-enabled flag (i.e., sps_reshaper_enabled_flag) is parsed in the SPS, the tile group header may parse additional data (i.e., information included in Table 13 or Table 14 above) for constructing the lookup table (FwdLUT and / or InvLUT). To do this, the state of the SPS reshaper flag (sps_reshaper_enabled_flag) may be first checked in the slice header or tile group header. When sps_reshaper_enabled_flag is true (or 1), an additional flag, i.e., tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag), may be parsed. The purpose of tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) may be to indicate the presence of a reshaping model. For example, when tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) is true (or 1), it may indicate that a reshaper exists for the current tile group (or current slice). When tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) is false (or 0), it may indicate that a reshaper does not exist for the current tile group (or current slice).

[0226] If a reshaper exists and is enabled in the current tile group (or current slice), the reshaper model (i.e., tile_group_reshaper_model() or slice_reshaper_model()) may be processed. Furthermore, an additional flag tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) may also be parsed. The tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) may indicate whether the reshaping model is used for the current tile group (or slice). For example, if tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is 0 (or false), it may indicate that the reshaping model is not used for the current tile group (or current slice). If tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is 1 (or true), it may indicate that the reshaping model is used for the current tile group (or slice).

[0227] As an example, tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) may be true (or 1) and tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) may be false (or 0). This means that the reshaping model exists but is not used in the current tile group (or slice). In this case, the reshaping model may be used in future tile groups (or slices). As another example, tile_group_reshaper_enable_flag may be true (or 1) and tile_group_reshaper_model_present_flag may be false (or 0). In this case, the decoder uses the reshaper from the previous initialization.

[0228] When parsing the reshaping model (i.e., tile_group_reshaper_model() or slice_reshaper_model()) and tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag), it may be determined (evaluated) whether conditions required for chroma scaling exist. The above conditions include condition 1 (the current tile group / slice has not been intra-coded) and / or condition 2 (the current tile group / slice has not been partitioned into two separate coding quadtree structures for luma and chroma, i.e., the block structure of the current tile group / slice is not a dual-tree structure). If condition 1 and / or condition 2 are true and / or tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is true (or 1), tile_group_reshaper_chroma_residual_scale_flag (or slice_reshaper_chroma_residual_scale_flag) may be parsed. When tile_group_reshaper_chroma_residual_scale_flag (or slice_reshaper_chroma_residual_scale_flag) is enabled (if 1 or true), it may indicate that chroma residual scaling is enabled for the current tile group (or slice). When tile_group_reshaper_chroma_residual_scale_flag (or slice_reshaper_chroma_residual_scale_flag) is disabled (if 0 or false), it may indicate that chroma residual scaling is disabled for the current tile group (or slice).

[0229] The purpose of the tile group shaping model is to resolve the data required to construct a lookup table (LUT). The idea behind the construction of these LUTs is that the distribution of the allowed range of luma values ​​can be divided into a number of bins (e.g., 16 bins), which can be represented using a set of 16 PWL equations. Therefore, any luma value that falls within a given bin can be mapped to a modified luma value.

[0230] Fig.11 A diagram showing an exemplary forward mapping is shown. Fig.11 In FIG, five bins are shown by way of example.

[0231] Reference Fig.11, the x-axis represents the input luminance value and the y-axis represents the modified output luminance value. The x-axis is divided into 5 bins or slices, each of which has a length of L. That is, the five bins mapped to the modified luminance value have the same length. The forward lookup table (FwdLUT) can be constructed using the data available from the tile group header (i.e., the shaper data), so the mapping can be facilitated.

[0232] In one embodiment, an output pivot point associated with a bin index may be calculated. The output pivot point may set (mark) the minimum and maximum boundaries of the output range for luma codeword shaping. The calculation process of the output pivot point may be performed by calculating a piecewise cumulative distribution function (CDF) of the number of codewords. The output pivot range may be sliced ​​based on the maximum number of bins to be used and the size of the lookup table (FwdLUT or InvLUT). As an example, the output pivot range may be sliced ​​based on the product between the maximum number of bins and the size of the lookup table (size of the LUT * maximum number of bin indices). For example, if the product between the maximum number of bins and the size of the lookup table is 1024, the output pivot range may be sliced ​​into 1024 entries. This sawtoothing of the output pivot range may be performed (applied or implemented) based on (using) a scaling factor. In one example, the scaling factor may be derived based on Equation 1 below.

[0233] [Formula 1]

[0234] SF=(y2-y1)*(1< <FP_PREC)+c

[0235] In Formula 1, SF represents a scaling factor, and y1 and y2 represent output pivot points corresponding to respective bins. In addition, FP_PREC and c may be predetermined constants. The scaling factor determined based on Formula 1 may be referred to as a scaling factor for forward shaping.

[0236] In another embodiment, for reverse shaping (reverse mapping), for the defined range of bins to be used (i.e., from reshaper_model_min_bin_idx to reshape_model_max_bin_idx), the input shaping pivot point and the mapping reverse output pivot point corresponding to the mapping pivot point of the forward LUT are obtained (given by the bin index considered * the number of initial codewords). In another example, the scaling factor SF can be derived based on the following formula 2.

[0237] [Formula 2]

[0238] SF=(y2-y1)*(1< <FP_PREC) / (x2-x1)

[0239] In Formula 2, SF represents a scaling factor, x1 and x2 represent input pivot points, and y1 and y2 represent output pivot points (reverse mapped output pivot points) corresponding to each fragment (bin). Here, the input pivot point may be a pivot point mapped based on a forward lookup table (FwdLUT), and the output pivot point may be a pivot point reversely mapped based on an inverse lookup table (InvLUT). In addition, FP_PREC may be a predetermined constant value. The FP_PREC of Formula 2 may be the same as or different from the FP_PREC of Formula 1. The scaling factor determined based on Formula 2 may be referred to as a scaling factor for reverse shaping. During reverse shaping, the segmentation of the input pivot point may be performed based on the scaling factor of Formula 2. The scaling factor SF is used to slice the range of the input pivot point. Based on the segmented input pivot point, bin indices ranging from 0 to minimum bin index (reshaper_model_min_bin_idx) and / or from minimum bin index (reshaper_model_min_bin_idx) to maximum bin index (reshape_model_max_bin_idx) are assigned pivot values ​​corresponding to minimum bin value and maximum bin value.

[0240] In one example, LMCS data (lmcs_data) may be included in the APS. For example, the semantics of the APS may be 32 APSs signaled for encoding.

[0241] The following table shows the syntax and semantics of an exemplary APS according to embodiments of this document.

[0242] [Table 15]

[0243]

[0244] [Table 16]

[0245]

[0246] Referring to Table 15, type information (eg, aps_params_type) of an APS parameter may be parsed / signaled in an APS. Type information of an APS parameter may be parsed / signaled after adaptation_parameter_set_id.

[0247] APS_params_type, ALF_APS, and LMCS_APS included in the above Table 15 may be described according to the following table. That is, according to APS_params_type included in Table 15, the type of APS parameters applied to APS may be set as shown in Table 3.2 included in Table 16.

[0248] Referring to Table 16, for example, aps_params_type may be a syntax element for classifying the type of the corresponding APS parameter. The type of the APS parameter may include an ALF parameter and an LMCS parameter. Referring to Table 16, when the value of the type information (aps_params_type) is 0, the name of aps_params_type may be determined as ALF_APS (or ALF APS), and the type of the APS parameter may be determined as an ALF parameter (the APS parameter may represent the ALF parameter). In this case, the ALF data field (i.e., alf_data()) may be parsed / signaled to the APS. When the value of the type information (aps_params_type) is 1, the name of aps_params_type may be determined as LMCS_APS (or LMCS APS), and the type of the APS parameter may be determined as an LMCS parameter (the APS parameter may represent the LMCS parameter). In this case, LMCS (shaper model, shaper) data (i.e., lmcs_data()) may be parsed / signaled to the APS.

[0249] Table 17 and / or Table 18 below illustrate the syntax of the reshaper model according to an embodiment. The reshaper model may be referred to as a LMCS model. Although the reshaper model is exemplarily described herein as a tile group reshaper, the present specification is not necessarily limited to this embodiment. For example, the reshaper model may be included in the APS, or the tile group reshaper model may be referred to as a slice reshaper model or LMCS data (LMCS data field). In addition, the prefix "reshaper_model" or "Rsp" may be used interchangeably with "lmcs". For example, in the following tables and descriptions, reshaper_model_min_bin_idx, reshaper_model_delta_max_bin_idx, reshaper_model_max_bin_idx, RspCW, and RsepDeltaCW may be used interchangeably with lmcs_min_bin_idx, lmcs_delta_cs_bin_idx, lmx mixed, lmcs_delta_csDcs_bin_idx, and Wlm_idx, respectively.

[0250] The LMCS data (lmcs_data()) or the shaper model (tile group shaper or slice shaper) included in the above Table 15 can be expressed as the syntax included in the following table.

[0251] [Table 17]

[0252]

[0253] [Table 18]

[0254]

[0255] The semantics of the syntax elements included in the syntax of Table 17 and / or Table 18 may include, for example, matters disclosed in the following table.

[0256] [Table 19]

[0257]

[0258]

[0259] [Table 20]

[0260]

[0261]

[0262] The reverse mapping process of the brightness samples according to this document can be described in the form of standard documents as shown in the following table.

[0263] [Table 21]

[0264]

[0265] The identification of the piecewise function index processing of luma samples according to this document can be described in the form of standard documents as shown in the following table: In Table 22, idxYInv can be called a reverse mapping index, and the reverse mapping index can be derived based on the reconstructed luma sample (lumaSample).

[0266] [Table 22]

[0267]

[0268]

[0269] Luma mapping may be performed based on the above embodiments and examples, and the above syntax and components included therein may be merely exemplary representations, and the embodiments in this document are not limited to the above tables or formulas. Hereinafter, a method for performing chroma residual scaling (scaling of the chroma component of the residual sample) based on luma mapping is described.

[0270] (Lumence-dependent) chroma residual scaling is designed to compensate for the interaction between a luma signal and its corresponding chroma signal. For example, chroma residual scaling is also signaled at the tile group level whether it is enabled. In one example, if luma mapping is enabled and if dual-tree partitioning (also called separate chroma tree) is not applied for the current tile group, an additional flag is signaled to indicate whether luma-dependent chroma residual scaling is enabled. In other examples, luma-dependent chroma residual scaling is disabled when luma mapping is not used, or when dual-tree partitioning is used in the current tile group. In another example, luma-dependent chroma residual scaling is always disabled for chroma blocks with an area less than or equal to 4.

[0271] The chroma residual scaling may be based on the average value of the corresponding luma prediction block (the luma component of the prediction block to which the intra prediction mode and / or inter prediction mode is applied). The scaling operation at the encoder and / or decoder side may be implemented in fixed-point integer arithmetic based on the following equation 3.

[0272] [Formula 3]

[0273] c'=sign(c)*((abs(c)*s+2CSCALE_FP_PREC-1)>>CSCALE_FP_PREC)

[0274] In Equation 3, c' represents the scaled chroma residual sample (the scaled chroma component of the residual sample), c represents the chroma residual sample (the chroma residual sample, the chroma component of the residual sample), s represents the chroma residual scaling factor, and CSCALE_FP_PREC represents a (predefined) constant value to specify the precision. For example, CSCALE_FP_PREC may be 11.

[0275] Fig.12 is a flowchart illustrating a method for deriving a chroma residual scaling index according to an embodiment of the present document. Fig.12 The method in can be based on Fig. 9 and included in Fig. 9 The tables, formulas, variables, arrays and functions in the relevant description are executed.

[0276] In step S1210, it may be determined whether the prediction mode of the current block is an intra prediction mode or an inter prediction mode based on the prediction mode information. If the prediction mode is an intra prediction mode, the current block or the prediction sample of the current block is considered to be already in the shaped (mapped) area. If the prediction mode is an inter prediction mode, the current block or the prediction sample of the current block is considered to be in the original (unmapped, unshaped) area.

[0277] In step S1220, when the prediction mode is the intra prediction mode, the average of the current block (or the brightness prediction samples of the current block) may be calculated (derived). That is, the average of the current block in the shaped area is directly calculated. The average may also be referred to as an average value or mean.

[0278] In step S1221, when the prediction mode is an inter prediction mode, forward shaping (forward mapping) may be performed (applied) on the luma prediction samples of the current block. Through forward shaping, the luma prediction samples based on the inter prediction mode may be mapped from the original region to the shaping region. In one example, the forward shaping of the luma prediction samples may be performed based on the shaping model described in Table 17 and / or Table 18 above.

[0279] In step S1222, the average of the forward-shaped (forward-mapped) luma prediction samples may be calculated (derived). That is, an average process of the forward-shaped results may be performed.

[0280] In step S1230, a chroma residual scaling index may be calculated. When the prediction mode is an intra prediction mode, the chroma residual scaling index may be calculated based on an average of luma prediction samples. When the prediction mode is an inter prediction mode, the chroma residual scaling index may be calculated based on an average of forward-shaped luma prediction samples.

[0281] In an embodiment, the chroma residual scaling index may be calculated based on a for loop syntax. The following table shows an exemplary for loop syntax for deriving (calculating) the chroma residual scaling index.

[0282] [Table 23]

[0283]

[0284] In Table 23, idxS represents the chroma residual scaling index, idxFound represents the index of the chroma residual scaling index that identifies whether the condition of the if statement is obtained, S represents a predetermined constant value, and MaxBinIdx represents the maximum allowed bin index. ReshapPivot[idxS+1] (in other words, LmcsPivot[idxS+1]) can be derived based on Table 19 and / or Table 20 above.

[0285] In an embodiment, the chroma residual scaling factor may be derived based on the chroma residual scaling index. Equation 4 is an example for deriving the chroma residual scaling factor.

[0286] [Formula 4]

[0287] s = ChromaScaleCoef[idxS]

[0288] In Formula 4, s represents the chroma residual scaling factor, and ChromaScaleCoef can be a variable (or array) derived based on Table 19 and / or Table 20 above.

[0289] As described above, an average luma value of the reference sample may be obtained, and a chroma residual scaling factor may be derived based on the average luma value. As described above, chroma component residual samples may be scaled based on the chroma residual scaling factor, and chroma component reconstructed samples may be generated based on the scaled chroma component residual samples.

[0290] In one embodiment of the present document, a signaling structure for efficiently applying the above-mentioned LMCS is proposed. According to this embodiment of the present document, for example, LMCS data may be included in HLS (ie, APS), and through lower-level header information (ie, picture header, slice header) of the APS, the LMCS model (shaper model) may be adaptively derived by signaling the ID of the APS (referred to as header information). The LMCS model may be derived based on LMCS parameters. In addition, for example, multiple APS IDs may be signaled through header information, and thus, different LMCS models may be applied in units of blocks within the same picture / slice.

[0291] In one embodiment according to the present document, a method is proposed to efficiently perform the operations required for LMCS. According to the semantics described above in Table 19 and / or Table 20, a division operation by the segment length lmcsCW[i] (also referred to as RspCW[i] in this document) is required to derive InvScaleCoeff[i]. The segment length of the reverse mapping may not be a power of 2, which means that the division cannot be performed by bit shifting.

[0292] For example, calculating InvScaleCoeff may require up to 16 divisions per slice. According to Table 19 and / or Table 20 above, for 10-bit encoding, lmcsCW[i] ranges from 8 to 511, so in order to implement the division operation according to lmcsCW[i] using the LUT, the size of the LUT must be 504. In addition, for 12-bit encoding, lmcsCW[i] ranges from 32 to 2047, so the LUT size needs to be 2016 to implement the division operation according to lmcsCW[i] using the LUT. That is, division is expensive in hardware implementation, so it is desirable to avoid division as much as possible.

[0293] In one aspect of this embodiment, lmcsCW[i] may be constrained to be a multiple of a fixed number (or a predetermined number). Thus, the lookup table (LUT) for division (the capacity or size of the LUT) may be reduced. For example, if lmcsCW[i] becomes a multiple of 2, the size of the LUT for the replacement division process may be reduced by half.

[0294] In another aspect of this embodiment, it is proposed that for encoding at higher internal bit depths, on top of the existing constraint that "the value of lmcsCW[i] should be in the range of (OrgCW>>3) to (OrgCW<<3-1)", if the encoding bit depth is higher than 10, lmcsCW[i] is further constrained to be a multiple of 1<<(BitDepthY-10). Here, BitDepthY can be the luma bit depth. Therefore, the possible number of lmcsCW[i] does not change with the encoding bit depth, and the size of the LUT required to calculate InvScaleCoeff does not increase due to the higher encoding bit depth. For example, for a 12-bit internal encoding bit depth, the limit value of lmcsCW[i] is a multiple of 4, then the LUT for the replacement division process will be the same as that for 10-bit encoding. This aspect can be implemented alone, but can also be implemented in combination with the above aspects.

[0295] In another aspect of this embodiment, lmcsCW[i] may be constrained to a narrower range. For example, lmcsCW[i] may be constrained to be in the range from (OrgCW>>1) to (OrgCW<<1)-1. Then for 10-bit encoding, the range of lmcsCW[i] may be [32, 127], thus only a LUT of size 96 is needed to compute InvScaleCoeff.

[0296] In another aspect of this embodiment, lmcsCW[i] may be approximated to the nearest power of 2 and used in the shaper design. Thus, the division in the reverse mapping may be performed (and replaced) by a bit shift.

[0297] In one embodiment according to the present document, a constraint on the range of LMCS codewords is proposed. According to Table 8 above, the value of the LMCS codeword is in the range from (OrgCW>>3) to (OrgCW<<3)-1. This codeword range is too wide. When there is a large difference between RspCW[i] and OrgCW, visual artifacts may occur.

[0298] According to one embodiment of the present document, it is proposed to constrain the codewords of LMCS PWL mapping to a narrow range. For example, the range of lmcsCW[i] may be in the range of (OrgCW>>1) to (OrgCW<<1)-1.

[0299] In one embodiment according to the present document, it is proposed to use a single chroma residual scaling factor for chroma residual scaling in LMCS. Existing methods for deriving chroma residual scaling factors use the average value of the corresponding luminance block and derive the slope of each segment of the inverse luminance mapping as the corresponding scaling factor. In addition, the process of identifying the segment index requires the availability of the corresponding luminance block, which leads to delay problems. This is not desirable for hardware implementation. According to this embodiment of the present document, the scaling in the chroma block may not depend on the luminance block value, and it may not be necessary to identify the segment index. Therefore, the chroma residual scaling process in LMCS can be performed without delay problems.

[0300] In one embodiment according to the present document, a single chroma scaling factor may be derived from both the encoder and the decoder based on luma LMCS information. When the LMCS luma model is received, the chroma residual scaling factor may be updated. For example, when the LMCS model is updated, the single chroma residual scaling factor may be updated.

[0301] The following table shows an example of obtaining a single chroma scaling factor according to the present embodiment.

[0302] [Table 24]

[0303]

[0304] Referring to Table 24, a single chroma scaling factor (eg, ChromaScaleCoeff or ChromaScaleCoeffSingle) may be obtained by averaging the inverse luma mapping slopes of all segments within LMCS_min_bin_idx and lmcs_max_bin_idx.

[0305] Fig.13 A linear fit of the pivot point according to an embodiment of the present document is shown. Fig.13 In FIG. 1 , pivot points P1, Ps and P2 are shown. Fig.13 to describe.

[0306] In an example of this implementation, a single chroma scaling factor may be obtained based on a linear approximation of the luma PWL mapping between the pivot points lmcs_min_bin_idx and lmcs_max_bin_idx+1 (LmcsMaxBinIdx+1). That is, the inverse slope of the linear mapping may be used as the chroma residual scaling factor. For example, Fig.13 The linear line 1 may be a straight line connecting the pivot points P1 and P2. Fig.13, in P1, the input value is x1 and the mapped value is 0, and in P2, the input value is x2 and the mapped value is y2. The inverse slope (inverse scale) of Linear Line 1 is (x2-x1) / y2, and a single chroma scaling factor ChromaScaleCoeffSingle can be calculated based on the input and mapped values ​​of the pivot points P1, P2 and the following formula.

[0307] [Formula 5]

[0308] ChromaScaleCoeffSingle=(x2-x1)*(1< <CSCALE_FP_PREC) / y2

[0309] In Formula 5, CSCALE_FP_PREC represents a shift factor, for example, CSCALE_FP_PREC may be a predetermined constant value. In one example, CSCALE_FP_PREC may be 11.

[0310] In another example according to this embodiment, referring to Fig.13 , the input value at the pivot point Ps is min_bin_idx+1, and the mapped value at the pivot point Ps is ys. Therefore, the inverse slope (inverse scale) of the linear line 1 can be calculated as (xs-x1) / ys, and a single chroma scaling factor ChromaScaleCoeffSingle can be calculated based on the input value and mapped value of the pivot point P1, Ps and the following formula.

[0311] [Formula 6]

[0312] ChromaScaleCoeffSingle=(xs-x1)*(1< <CSCALE_FP_PREC) / ys

[0313] In Formula 6, CSCALE_FP_PREC represents a shift factor (bit shift factor), for example, CSCALE_FP_PREC may be a predetermined constant value. In one example, CSCALE_FP_PREC may be 11, and the de-scaling bit shift may be performed based on CSCALE_FP_PREC.

[0314] In another example according to this embodiment, a single chroma residual scaling factor may be derived based on a linear approximation line. An example of a derived linear approximation line may include a linear connection of pivot points (i.e., lmcs_min_bin_idx, lmcs_max_bin_idx+1). For example, the linear approximation result may be represented by a codeword mapped by PWL. The mapped value y2 at P2 may be the sum of the codewords of all bins (shards), and the difference (x2-x1) between the input value at P2 and the input value at P1 is OrgCW*(lmcs_max_bin_idx-lmcs_min_bin_idx+1) (for OrgCW, see Table 19 and / or Table 20 above). The following table shows an example of obtaining a single chroma scaling factor according to the above embodiment.

[0315] [Table 25]

[0316]

[0317] Referring to Table 25, a single chroma scaling factor (eg, ChromaScaleCoeffSingle) may be obtained from two pivot points (ie, lmcs_min_bin_idx, lmcs_max_bin_idx). For example, the inverse slope of a linear mapping may be used as the chroma scaling factor.

[0318] In another example of this embodiment, a single chroma scaling factor may be obtained by a linear fit of the pivot point to minimize the error (or mean square error) between the linear fit and the existing PWL mapping. This example may be more accurate than simply connecting the two pivot points at lmcs_min_bin_idx and lmcs_max_bin_idx. There are many ways to find the optimal linear mapping, and such an example is described below.

[0319] In one example, parameters b1 and b0 of the linear fitting equation y=b1*x+b0 for minimizing the sum of the least squares errors may be calculated based on the following equation 7 and / or equation 8.

[0320] [Formula 7]

[0321]

[0322] [Formula 8]

[0323]

[0324] In equations 7 and 8, x is the original brightness value, y is the shaped brightness value, and is the mean of x and y, and xi and yi represent the value of the i-th pivot point.

[0325] Reference Fig.13, another simple approximation for identifying linear mappings is given by:

[0326] - Linear line 1 is obtained by connecting the pivot points of the PWL map at lmcs_min_bin_idx and lmcs_max_bin_idx+1, and this linear line is calculated with input values ​​that are multiples of OrgW lmcs_pivots_linear[i]

[0327] -Use Linear Line 1 and sum the differences of the pivot point mapping values ​​using the PWL mapping.

[0328] -Get the average difference avgDiff.

[0329] - Adjust the last pivot point of the Linear Line by the average difference, e.g. 2*avgDiff

[0330] - Use the inverse slope of the adjusted linear line as the chromaticity residual scale.

[0331] According to the above linear fitting, the chroma scaling factor (ie, the inverse slope of the forward mapping) can be derived (obtained) based on the following Equation 9 or Equation 10.

[0332] [Formula 9]

[0333] ChromaScaleCoeffSingle=OrgCW*(1< <CSCALE_FP_PREC) / lmcs_pivots_linear[lmcs_min_bin_idx+1]

[0334] [Formula 10]

[0335] ChromaScaleCoeffSingle=OrgCW*(lmcs_max_bin_idx-lmcs_max_bin_idx+1)*(1< <CSCALE_FP_PREC) / lmcs_pivots_linear[lmcs_max_bin_idx+1]

[0336] In the above formula, lmcs_pivots_linear[i] may be a mapping value of a linear mapping. For a linear mapping, all segments of a PWL mapping between a minimum bin index and a maximum bin index may have the same LMCS codeword (lmcsCW). That is, lmcs_pivots_linear[lmcs_min_bin_idx+1] may be the same as lmcsCW[lmcs_min_bin_idx].

[0337] In addition, in Equations 9 and 10, CSCALE_FP_PREC represents a shift factor (bit shift factor), and for example, CSCALE_FP_PREC may be a predetermined constant value. In one example, CSCALE_FP_PREC may be 11.

[0338] With a single chroma residual scaling factor (ChromaScaleCoeffSingle), it is no longer necessary to calculate the average of the corresponding luminance block and find the index in the PWL linear map to obtain the chroma residual scaling factor. Therefore, the coding efficiency using chroma residual scaling can be increased. This not only eliminates the dependency on the corresponding luminance block, solves the delay problem, but also reduces complexity.

[0339] The luminance-dependent chrominance residual scaling process of semantic and / or chrominance samples related to LMCS data according to the above-mentioned present embodiment can be described in a standard document format as shown in the following table.

[0340] [Table 26]

[0341]

[0342] [Table 27]

[0343]

[0344]

[0345] In another embodiment of the present document, the encoder may determine parameters related to a single chroma scaling factor and signal these parameters to the decoder. Using signaling, the encoder may utilize other information available at the encoder to derive the chroma scaling factor. This embodiment aims to eliminate the chroma residual scaling delay problem.

[0346] For example, another example of identifying a linear mapping to be used to determine the chroma residual scaling factor is given as follows:

[0347] - Calculate this linear line lmcs_pivots_linear[i] with input values ​​that are multiples of OrgW by connecting the pivot points of the PWL map at lmcs_min_bin_idx and lmcs_max_bin_idx+1

[0348] - Get a weighted sum of the differences of the mapped values ​​of the pivot point using those of the linear line 1 and the luma PWL mappings. The weights may be based on encoder statistics (e.g. a histogram of bins).

[0349] -Get the weighted average difference avgDiff.

[0350] - Adjust the last pivot point of Linear 1 by the weighted average difference, e.g. 2*avgDiff

[0351] - Use the inverse slope of the adjusted linear line to calculate the chromaticity residual scale.

[0352] The following table shows an example of syntax for signaling the y-value for chroma scaling factor derivation.

[0353] [Table 28]

[0354]

[0355] In Table 28, the syntax element lmcs_chroma_scale may specify a single chroma (residual) scaling factor (ChromaScaleCoeffSingle=lmcs_chroma_scale) for LMCS chroma residual scaling. That is, information about the chroma residual scaling factor may be directly signaled, and the signaled information may be derived as the chroma residual scaling factor. In other words, the value of the signaled information about the chroma residual scaling factor may be (directly) derived as the value of the single chroma residual scaling factor. Here, the syntax element lmcs_chroma_scale may be signaled together with other LMCS data (i.e., syntax elements related to the absolute value and sign of the codeword, etc.).

[0356] Alternatively, the encoder may signal only the necessary parameters to derive the chroma residual scaling factors at the decoder. To derive the chroma residual scaling factors at the decoder, the input value x and the mapped value y are needed. Since the x value is the bin length and is a known number, it does not need to be signaled. After all, only the y value needs to be signaled in order to derive the chroma residual scaling factors. Here, the y value can be the mapped value of any pivot point in the linear mapping (i.e., Fig.13 The mapping value of P2 or Ps in ).

[0357] The following table shows an example of signaling mapping values ​​used to derive the chroma residual scaling factor.

[0358] [Table 29]

[0359]

[0360] [Table 30]

[0361]

[0362] One of the syntaxes of Tables 29 and 30 above can be used to signal the y value at any linear pivot point specified by the encoder and decoder. That is, the encoder and decoder can use the same syntax to derive the y value.

[0363] First, an embodiment according to Table 29 is described. In Table 29, lmcs_cw_linear may represent a mapping value at Ps or P2. That is, in the embodiment according to Table 29, a fixed number may be signaled through lmcs_cw_linear.

[0364] In an example according to this embodiment, if lmcs_cw_linear represents the mapping value of one bin (ie, Fig.13 lmcs_pivots_linear[lmcs_min_bin_idx+1] in Ps), the chroma scaling factor can be derived based on the following formula.

[0365] [Formula 11]

[0366] ChromaScaleCoeffSingle=OrgCW*(1< <CSCALE_FP_PREC) / lmcs_cw_linear

[0367] In another example according to this embodiment, if lmcs_cw_linear represents lmcs_max_bin_idx+1 (ie, Fig.13 lmcs_pivots_linear[lmcs_max_bin_idx+1] in P2), the chroma scaling factor can be derived based on the following formula.

[0368] [Formula 12]

[0369] ChromaScaleCoeffSingle=OrgCW*(lmcs_max_bin_idx-lmcs_max_bin_idx+1)*(1< <CSCALE_FP_PREC) / lmcs_cw_linear

[0370] In the above formula, CSCALE_FP_PREC represents a shift factor (bit shift factor), for example, CSCALE_FP_PREC may be a predetermined constant value. In one example, CSCALE_FP_PREC may be 11.

[0371] Next, an embodiment according to Table 30 is described. In this embodiment, lmcs_cw_linear may be signaled as an incremental value relative to a fixed number (ie, lmcs_delta_abs_cw_linear, lmcs_delta_sign_cw_linear_flag). In this example of an embodiment, when lmcs_cw_linear represents lmcs_pivots_linear[lmcs_min_bin_idx+1] (ie, Fig.13Ps)

[0372] When the mapping values ​​in are used, lmcs_cw_linear_delta and lmcs_cw_linear can be derived based on the following formula.

[0373] [Formula 13]

[0374] lmcs_cw_linear_delta=(1-2*lmcs_delta_sign_cw_linear_flag)*lmcs_delta_abs_linear_cw

[0375] [Formula 14]

[0376] lmcscwl_inear=lmcs_cw_linear_delta+OrgCW

[0377] In another example of this embodiment, when lmcs_cw_linear represents lmcs_pivots_linear[lmcs_max_bin_idx+1]

[0378] (Right now, Fig.13 When the mapping value in P2) is used, lmcs_cw_linear_delta and lmcs_cw_linear can be derived based on the following formula.

[0379] [Formula 15]

[0380] lmcs_cw_linear_delta=(1-2*lmcs_delta_sign_cw_linear_flag)*lmcs_delta_abs_linear_cw

[0381] [Formula 16]

[0382] lmcs_cw_linear=1mcs_cw_linear_delta+OrgCW*(lmcs_max_bin_idx-lmcs_max_bin_idx+1)

[0383] In the above formula, OrgCW can be a value derived based on Table 19 and / or Table 20 above.

[0384] The luminance-dependent chrominance residual scaling process for semantic and / or chrominance samples associated with LMCS data according to the present embodiment described above can be described in a standard document format as shown in the following table.

[0385] [Table 31]

[0386]

[0387]

[0388] [Table 32]

[0389]

[0390] Fig.14 An example of linear shaping (or linear shaping, linear mapping) according to an embodiment of this document is shown. That is, in this embodiment, it is proposed to use a linear shaper in LMCS. For example, Fig.14 This example in may involve forward linear shaping (mapping).

[0391] Reference Fig.14 , the linear shaper may include two pivot points, namely, P1 and P2. P1 and P2 may represent input values ​​and mapped values, for example, P1 may be (min_input, 0) and P2 may be (max_input, max_mapped). Here, min_input represents the minimum input value and max_input represents the maximum input value. Any input value less than or equal to min_input is mapped to 0, and any input value greater than max_input is mapped to max_mapped. Any input brightness value within min_input and max_input is linearly mapped to other values. Fig.14 An example of a mapping is shown. The pivot points P1 , P2 may be determined at the encoder and a linear fit may be used to approximate a piecewise linear mapping.

[0392] In another embodiment according to the present document, another example of a method of signaling a linear shaper may be presented. The pivot points P1, P2 of the linear shaper model may be explicitly signaled. The following table shows an example of the syntax and semantics of explicitly signaling a linear shaper model according to this example.

[0393] [Table 33]

[0394]

[0395] [Table 34]

[0396]

[0397] Referring to Table 33 and Table 34, the input value of the first pivot point can be derived based on the syntax element lmcs_min_input, and the input value of the second pivot point can be derived based on the syntax element lmcs_max_input. The mapping value of the first pivot point can be a predetermined value (a value known to both the encoder and the decoder), for example, the mapping value of the first pivot point is 0. The mapping value of the second pivot point can be derived based on the syntax element lmcs_max_mapped. That is, the linear shaping model can be signaled explicitly (directly) based on the information signaled in the syntax of Table 33.

[0398] Alternatively, lmcs_max_input and lmcs_max_mapped can be signaled as incremental values. The following table shows an example of the syntax and semantics of signaling the linear shaping model as incremental values.

[0399] [Table 35]

[0400]

[0401] [Table 36]

[0402]

[0403]

[0404] Referring to Table 36, the input value of the first pivot point can be derived based on the syntax element lmcs_min_input. For example, lmcs_min_input can have a mapping value of 0. lmcs_max_input_delta can specify the difference between the input value of the second pivot point and the maximum luminance value (i.e., (1<<bitdepthY)-1). lmcs_max_mapped_delta can specify the difference between the mapping value of the second pivot point and the maximum luminance value (i.e., (1<<bitdepthY)-1).

[0405] According to an embodiment of the present document, forward mapping of luma prediction samples, reverse mapping of luma reconstruction samples, and chroma residual scaling may be performed based on the above-mentioned example of a linear shaper. In one example, reverse scaling of luma (reconstructed) samples (pixels) in reverse mapping based on a linear shaper may require only one reverse scaling factor. The same is true for forward mapping and chroma residual scaling. That is, the steps of determining ScaleCoeff[i], InvScaleCoeff[i], and ChromaScaleCoeff[i] (i is the bin index) may be replaced by only one single factor. Here, one single factor is a fixed point representation of the (positive) slope or inverse slope of the linear mapping. In one example, the reverse luma mapping scaling factor (the reverse scaling factor in the reverse mapping of luma reconstruction samples) may be derived based on at least one of the following equations.

[0406] [Formula 17]

[0407] InvScaleCoeffSingle=OrgCW / lmcsCWLinear

[0408] [Formula 18]

[0409] InvScaleCoeffSingle=OrgCW*(lmcs_max_bin_idx-lmcs_max_bin_idx+1) / lmcsCWLinearAll

[0410] [Formula 19]

[0411] InvScaleCoeffSingle=(lmcs_max_input-lmcs_min_input) / lmcsCWLinearAll

[0412] lmcsCWLinear of Formula 17 may be derived from the above Table 31. lmcsCWLinearALL of Formula 18 and Formula 19 may be derived from at least one of the above Tables 33 to 36. In Formula 17 or Formula 18, OrgCW may be derived from Table 19 and / or Table 20.

[0413] The following table describes the formula and syntax (conditional statement) indicating the forward mapping processing of luma samples (i.e., luma prediction samples) in picture reconstruction. In the following table and formula, FP_PREC is a constant value of bit shift and can be a predetermined value. For example, FP_PREC can be 11 or 15.

[0414] [Table 37]

[0415]

[0416] [Table 38]

[0417]

[0418] Table 37 can be used to derive the luminance samples of the forward mapping in the luminance mapping process based on Tables 17 to 20 above. That is, Table 37 can be described together with Tables 19 and 20. In Table 37, the luminance (prediction) samples PredMapPSamples[i][j] of the forward mapping as the output can be derived from the luminance (prediction) samples predSamples[i][j] as the input. The idxY of Table 37 can be referred to as the (forward) mapping index, and the mapping index can be derived based on the predicted luminance samples.

[0419] Table 38 can be used to derive the luminance samples of the forward mapping in the luminance mapping based on the linear shaper. For example, lmcs_min_input, lmcs_max_input, lmcs_max_mapped, and ScaleCoeffSingle of Table 38 can be derived from at least one of Tables 33 to 36. In Table 38, in the case of "lmcs_min_input < predSamples[i][j] < lmcs_max_input", the luminance (prediction) samples PredMapSamples[i][j] of the forward mapping can be derived from the input luminance (prediction) samples predSamples[i][j] as the output. Comparing between Table 37 and Table 38, the change with respect to the existing LMCS according to the application of the linear shaper can be seen from the perspective of the forward mapping.

[0420] The following formula and table describe the inverse mapping process of the luminance samples (i.e., luminance reconstruction samples). In the following formula and table, the "lumaSample" as the input can be the luminance reconstruction sample before the inverse mapping (before modification). The "invSample" as the output can be the (modified) luminance reconstruction sample of the inverse mapping. In other cases, the clipped invSample can be referred to as the modified luminance reconstruction sample.

[0421] [Equation 20]

[0422] invSample = InputPivot[idxYInv] + (InvScaleCoeff[idxYInv] * (lumaSample - LmcsPivot[idxYInv] + (1 << (FPPREC - 1)))) >> FP_PREC

[0423] [Equation 21]

[0424] invSample=lmcs_min_input+(InvScaleCoeffSingle*(lumaSample-lmcs_min_input)+(1<<(FP_PREC-1)))>>FP_PREC

[0425] [Table 39]

[0426]

[0427] [Table 40]

[0428]

[0429] Equation 21 may be used to derive inversely mapped luma samples in luma mapping according to the present document. In Equation 20, the index idxInv may be derived based on Table 50, Table 51, or Table 52 described later.

[0430] Equation 21 can be used to derive the reverse mapped luma samples from the luma mapping according to the application of the linear shaper. For example, lmcs_min_input of Equation 21 can be derived from at least one of Tables 33 to 36. By comparing Equation 20 with Equation 21, the changes relative to the existing LMCS according to the application of the linear shaper can be seen from the perspective of forward mapping.

[0431] Table 39 may include an example of an equation for deriving a reverse-mapped luma sample in luma mapping. For example, the index idxInv may be derived based on Table 50, Table 51, or Table 52 described later.

[0432] Table 40 may include other examples of equations for deriving inversely mapped luma samples in luma mapping. For example, lmcs_min_input and / or lmcs_max_mapped of Table 40 may be derived from at least one of Tables 33 to 36, and / or InvScaleCoeffSingle of Table 40 may be at least Tables 33 to 36, and / or Equations 17 to 19 may be derived from one.

[0433] Based on the above example of the linear shaper, the segment index identification process can be omitted. That is, in this example, since there is only one segment with valid shaped luma pixels, the segment index identification process for inverse luma mapping and chroma residual scaling can be removed. Therefore, the complexity of inverse luma mapping can be reduced. In addition, the delay problem caused by relying on luma segment index identification during chroma residual scaling can be eliminated.

[0434] According to the embodiments using the above linear shaper, the following advantages can be provided for LMCS: i) The encoder shaper design can be simplified, thereby preventing possible artifacts caused by sudden changes between piecewise linear segments; ii) The decoder inverse mapping process that can remove the piecewise index identification process can be simplified by eliminating the piecewise index identification process; iii) By removing the piecewise index identification process, the latency problem caused by dependence on the corresponding luminance block in chrominance residual scaling can be removed; iv) The signaling overhead can be reduced, and more frequent updates of the shaper can be made more feasible; v) For many places where loops with 16 segments were required in the past, the loops can be eliminated. For example, in order to derive InvScaleCoeff[i], the number of division operations according to lmcsCW[i] can be reduced to 1.

[0435] In another embodiment according to this document, a flexible bin-based LMCS is proposed. Here, a flexible bin may refer to a bin whose number is not fixed to a predetermined (predefined, specific) number. In the existing embodiments, the number of bins in LMCS is fixed to 16, and for the input sample values, these 16 bins are equally distributed. In this embodiment, a flexible number of bins is proposed, and these segments (bins) may not be equally distributed in terms of the original pixel values.

[0436] The following table exemplarily shows the syntax of the LMCS data (data field) according to this embodiment and the semantics of the syntax elements included therein.

[0437] [Table 41]

[0438]

[0439] [Table 42]

[0440]

[0441]

[0442] Referring to Table 41, the information lmcs_num_bins_minus1 regarding the number of bins can be signaled. Referring to Table 42, lmcs_num_bins_minus1 + 1 can be equal to the number of bins, and the number of bins can be in the range from 1 to (1 << BitDepthY)-1. For example, lmcs_num_bins_minus1 or lmcs_num_bins_minus1 + 1 can be a multiple of a power of 2.

[0443] In the implementation described together with Tables 41 and 42, regardless of whether the shaper is linear (signaling of lmcs_num_bins_minus1), the number of pivot points can be derived based on lmcs_num_bins_minus1 (information about the number of bins), and the input value and mapping value of the pivot point (LmcsPivot_input[i], LmcsPivot_mapped[i]) can be derived based on the sum of the signaled codeword values ​​(lmcs_delta_input_cw[i], lmcs_delta_mapped_cw[i]) (here, the initial input value LmcsPivot_input[0] and the initial output value LmcsPivot_mapped[0] are 0).

[0444] Fig.15 An example of linear forward mapping in an embodiment of this document is shown. Fig.16 An example of inverse forward mapping in the embodiment of this document is shown.

[0445] In accordance with Fig.15 and Fig.16 In an embodiment of the present invention, a method for supporting both conventional LMCS and linear LMCS is proposed. In an example according to this embodiment, conventional LMCS and / or linear LMCS may be indicated based on the syntax element lmcs_is_linear. In the encoder, after determining the linear LMCS line, the mapping value (i.e., Fig.15 and Fig.16 The codeword in bin LmcsMaxBinIdx may be signaled using the syntax of the LMCS data or shaper mode described above.

[0446] The following table exemplarily shows the syntax of LMCS data (data field) according to an example of this embodiment and the semantics of the syntax elements included therein.

[0447] [Table 43]

[0448]

[0449] [Table 44]

[0450]

[0451]

[0452] The following table exemplarily shows the syntax of LMCS data (data field) according to another example of this embodiment and the semantics of the syntax elements included therein.

[0453] [Table 45]

[0454]

[0455] [Table 46]

[0456]

[0457]

[0458] Referring to Tables 43 to 46, when lmcs_is_linear_flag is true, all LMCSDeltaCW[i] between lmcs_min_bin_idx and LmcsMaxBinIdx may have the same value. That is, lmcsCW[i] of all segments between lmcs_min_bin_idx and LmcsMaxBinIdx may have the same value. The scale and descaling and chroma scale of all segments between lmcs_min_bin_idx and lmcsMaxBinIdx may be the same. Then if linear_shaper is true, there is no need to derive the segment index, it can use the scale, descaling from only one segment.

[0459] The following table exemplarily shows the identification process of the segment index according to the present embodiment.

[0460] [Table 47]

[0461]

[0462] According to another embodiment of the present document, the application of conventional 16-segment PWL LMCS and linear LMCS may depend on a higher level syntax (ie, sequence level).

[0463] The following table exemplarily shows the syntax of the SPS according to the present embodiment and the semantics of the syntax elements included therein.

[0464] [Table 48]

[0465]

[0466] [Table 49]

[0467]

[0468] Referring to Table 48 and Table 49, enabling of the normal LMCS and / or the linear LMCS may be determined (signaled) by a syntax element included in the SPS. Referring to Table 48, based on the syntax element sps_linear_lmcs_enabled_flag, one of the normal LMCS or the linear LMCS may be used in units of sequences.

[0469] In addition, whether only linear LMCS or conventional LMCS or both are enabled may also depend on the profile level. In one example, for a particular profile (i.e., SDR profile), only linear LMCS may be allowed, for another profile (i.e., HDR profile), only conventional LMCS may be allowed, and for another profile, both conventional LMCS and / or linear LMCS may be allowed.

[0470] According to another embodiment of the present document, the LMCS segment index identification process can be used in the reverse luma mapping and chroma residual scaling. In this embodiment, the segment index identification process can be used for blocks where chroma residual scaling is enabled, and is also called for all luma samples in the shaping (mapping) domain. This embodiment aims to keep its complexity low.

[0471] The following shows the identification process (derivation process) of the existing piecewise function index.

[0472] [Table 50]

[0473]

[0474] In an example, in the segment index identification process, the input samples may be classified into at least two categories. For example, the input samples may be classified into three categories, first, second, and third categories. For example, the first category may represent samples (values) less than LmcsPivot[lmcs_min_bin_idx+1], the second category may represent samples (values) greater than or equal to LmcsPivot[LmcsMaxBinIdx] (values), and the third category may indicate samples (values) between LmcsPivot[lmcs_min_bin_idx+1] and LmcsPivot[LmcsMaxBinIdx].

[0475] In this embodiment, it is proposed to optimize the recognition process by eliminating the category classification. This is because the input of the segment index recognition process is the brightness value in the shaping (mapping) domain, and no value should exceed the mapping value at the pivot point lmcs_min_bin_idx and LmcsMaxBinIdx+1. Therefore, the conditional processing of classifying samples into categories in the existing segment index recognition process is unnecessary. For more details, a specific example will be described in the table below.

[0476] In the example according to this embodiment, the identification process included in Table 50 can be replaced by one of the following Tables 51 or 52. Referring to Tables 51 and 52, the first two categories of Table 50 can be removed, and for the last category, the boundary value (second boundary value or end point) in the iterative for loop is changed from LmcsMaxBinIdx to LmcsMaxBinIdx+1. That is, the identification process can be simplified, and the complexity of the segment index derivation can be reduced. Therefore, LMCS-related encoding can be efficiently performed according to this embodiment.

[0477] [Table 51]

[0478]

[0479] [Table 52]

[0480]

[0481] Referring to Table 51, the comparison process corresponding to the condition of the if statement (the formula corresponding to the condition of the if statement) may be iteratively performed on all bin indices from the minimum bin index to the maximum bin index. When the formula corresponding to the condition of the if statement is true, the bin index may be derived as a reverse mapping index for reverse luma mapping (or a reverse scaling index for chroma residual scaling). Based on the reverse mapping index, a modified reconstructed luma sample (or a scaled chroma residual sample) may be derived.

[0482] According to Table 51, problems that may arise in the process of identifying (reverse) piecewise function indexes can be solved. Due to the implementation of Table 51, calculation loopholes can be removed and / or overlapping calculation processing due to overlapping boundary conditions can be omitted. If this document does not follow the implementation of Table 51, the mapping value used in the LMCS regarding the current block may exceed (deviate from) LmcsPivot[idxYInv+1] in the for syntax (loop syntax) used to identify the segment index (reverse mapping index). According to this embodiment, a mapping value within an appropriate range used in the LMCS regarding the current block can be used for the index identification process.

[0483] According to an embodiment of the present document, explicit chroma residual scaling factor signaling or implicit (indirect) chroma residual scaling factor derivation may be applied (performed). Flag information showing (indicating) whether explicit chroma residual scaling factor signaling or implicit chroma residual scaling factor derivation is applied may be included in the SPS. That is, the flag information may be an SPS level flag. In this embodiment, the LMCS model (LMCS data field) may be signaled in the APS.

[0484] In an embodiment, in the explicit chroma residual scaling factor derivation, information about the value of the chroma residual scaling factor may be explicitly signaled from the encoder to the decoder. For example, information about the value of the chroma residual scaling factor may be signaled in units of n×m blocks (CTUs). Here, n and / or m may be predefined positive integers. The values ​​of n and / or m may be signaled in a slice header. Thus, the granularity (regarding signaling and application) of the value of the chroma residual scaling factor may be controlled (from slice units to CTU (block) units). In explicit signaling, the value of the chroma residual scaling factor may be decoded in units of n×m CTUs. Chroma residual scaling may be applied to use the value of the chroma residual scaling factor decoded for all chroma blocks belonging to an area of ​​the n×m CTU. For example, the encoding process of the value of the chroma residual scaling factor may be substantially the same or similar to the encoding process of the incremental quantization parameter (QP).

[0485] In one example, in the implicit chroma residual scaling factor derivation, the chroma residual scaling factor may be derived based on the luma component (luma sample of the current block) of the current block (e.g., a block belonging to the VPDU level). In the implicit derivation, the chroma residual scaling factor of each block (VPDU) may be derived based on (the value of) the luma sample included in the previously encoded neighboring block (VPDU). In order to derive the chroma residual scaling factor of each block, the left neighboring reconstructed luma sample and / or the upper neighboring reconstructed luma sample may be used.

[0486] Fig.17 is a diagram showing the LMCS process according to an embodiment of the present document. Fig.17 Describes the derivation of the implicit chroma residual scaling factor.

[0487] Reference Fig.17 , the luma samples used to derive the chroma residual scaling factor may include M left-adjacent luma samples and M upper luma samples. In one example, the M left-adjacent luma samples and / or the M upper luma samples may be close to the position of the upper left sample. For example, M may be 16. Fig.16 As shown, the average value of M left adjacent luma samples and M upper luma samples can be derived (calculated) to derive the chroma residual scaling factor. Fig.11 The chroma residual scaling factor derived based on the average value of the luma samples is described in detail in the related description. The chroma residual scaling factor derived for a block (VPDU) can be used for all chroma blocks (samples) included in a block. According to the above-mentioned embodiment, encoding of the current block can be performed without requiring additional dependencies between luma samples and chroma samples.

[0488] When the number of upper neighboring samples is less than M for a block included in the rightmost column at a picture boundary, the picture boundary samples may be padded up to M. When the number of left neighboring samples is less than M for a block included in the bottom row, the picture boundary samples may be padded up to M. Therefore, the number of samples averaged may always be 2M (e.g., M is a multiple of 2), and thus, division may be omitted. For a block included in the leftmost column, only M upper neighboring samples may be used to derive an average value. Furthermore, for a block included in the top row, only M left neighboring samples may be used to derive an average value. In addition, for an upper left block of a picture, the average value may be set to a predetermined value, and, for example, the predetermined value may be 1<<(bitdepth-1).

[0489] The following table shows an exemplary syntax of an SPS according to the present embodiment.

[0490] [Table 53]

[0491]

[0492] Referring to Table 53, the SPS may include an explicit signaling enable flag sps_cs_explicit_flag showing (indicating) whether explicit chroma residual scaling factor signaling or implicit chroma residual scaling factor derivation is applied and / or an LMCS enable flag sps_lmcs_enabled_flag showing whether the LMCS data field is enabled. Based on the value of the LMCS enable flag being 1, the explicit signaling enable flag may be included in the SPS.

[0493] The following table shows an exemplary syntax of a slice header according to the present embodiment.

[0494] [Table 54]

[0495]

[0496] Referring to Table 54, the header information may include an LMCS enable flag slice_lmcs_enabled_flag showing whether the LMCS data field in the current slice (or current picture) is enabled. The LMCS enable flag included in the SPS may be referred to as a first LMCS enable flag, and the LMCS enable flag included in the header information may be referred to as a second LMCS enable flag. For example, when the value of the first LMCS enable flag is 1, the second LMCS enable flag may be included in the header information. When the value of the second LMCS enable flag is 1, the header information may include LMCS APS related ID information slice_lmcs_aps_id showing the ID of the LMCS APS referenced by the current picture and a chroma residual scaling enable flag slice_chroma_residual_scale_flag. The header information may be picture header information. That is, the syntax in Table 54 may be applied to information about the picture header, and in this case, the prefix slice_ may be replaced by ph_.

[0497] In one example, the header information may include a condition based on an explicit signaling enable flag sps_cs_explicit_flag and / or a chroma residual scaling enable flag slice_chroma_residual_scale_flag. When the condition that both the explicit signaling enable flag and the chroma residual scaling enable flag are true (or 1) is true, the header information may include information about the number of blocks sharing the same chroma residual scaling index (e.g., slice_cs_horiz_size_minus1, slice_cs_vertic_size_minus1).

[0498] The following table shows the semantics of the syntax elements included in Table 54.

[0499] [Table 55]

[0500]

[0501] Referring to Table 55, the information about the number of blocks sharing the same chroma residual scaling index may include information slice_cs_horiz_size_minus1 about the horizontal number of blocks sharing the same chroma residual scaling index and / or information slice_cs_vertic_size_minus1 about the vertical number of blocks sharing the same chroma residual scaling index. The horizontal number CsVerticSize of blocks sharing the same chroma residual scaling index may be derived based on the syntax element slice_cs_horiz_size_minus1, and the vertical number CsVerticSize of blocks sharing the same chroma residual scaling index may be derived based on the syntax element slice_cs_vertic_size_minus1. In this case, the horizontal number of blocks may mean the horizontal length of the area to which the blocks sharing the same chroma residual scaling index belong, and the vertical number of blocks may mean the vertical length of the area to which the blocks sharing the same chroma residual scaling index belong. In an example, the area to which the blocks sharing the same chroma residual scaling index belong may be the product of the horizontal number and the vertical number.

[0502] The following table shows an exemplary syntax of coding (tree) unit information according to the present embodiment.

[0503] [Table 56]

[0504]

[0505] Referring to Table 56, the coding unit information may include a condition based on an explicit signaling enable flag sps_cs_explicit_flag and / or a chroma residual scaling enable flag slice_chroma_residual_scale_flag. When the condition that both the explicit signaling enable flag and the chroma residual scaling enable flag are true (or 1) is true, the coding unit information may include information about the value of the chroma residual scaling factor. The information about the value of the chroma residual scaling factor may include information about the absolute value of the chroma residual scaling delta value cu_crs_delta_abs and / or information about the sign of the chroma residual scaling delta value cu_crs_delta_sign_flag. For example, when the value of the information about the absolute value of the chroma residual scaling delta value is 1 (or the value of the information about the absolute value of the chroma residual scaling delta value is greater than 0), the information about the sign of the chroma residual scaling delta value may be included in the coding unit information.

[0506] As a result, according to an embodiment of the present document, integrated chroma residual scaling can be applied to both dual-tree architectures and single-tree architectures. A high-level syntax (e.g., SPS) may include explicit signaling or implicit derivation enabling information about the chroma residual scaling factor. Special granularity can be controlled from CTU scale to slice scale by explicit signaling. Implicit derivation can be applied to each current block using adjacent reconstructed luma samples included in the current block VPDU. According to this embodiment, pipeline delays in chroma residual scaling can be removed. In addition, luma-chroma dependencies can be removed by explicit signaling. That is, information about the value of the explicit chroma residual scaling factor can be explicitly signaled, and therefore, the dependency between luma samples and chroma samples can be removed.

[0507] The following figures are constructed to describe the detailed examples of this document. The detailed names of the devices or the detailed names of the signals / messages / fields described in the figures are provided only as examples, and the technical features of this specification are not limited to the detailed names used in the following figures.

[0508] Fig.18 and Fig.19 An example of a video / image encoding method and related components according to an embodiment of the present document are schematically shown. Fig.18 The method shown in Figure 2 The encoding device disclosed in is executed. In detail, for example, Fig.18 S1800 and S1801 may be performed by the predictor of the encoding device, S1820 may be performed by the residual processor 230 of the encoding device, S1830 may be performed by the predictor 220 or the residual processor 230 of the encoding device, S1840 may be performed by the residual processor 230 or the adder 250 of the encoding device, S1850 may be performed by the residual processor 230 of the encoding device, and S1860 may be performed by the entropy encoder 240 of the encoding device. Fig.18 The method disclosed in may include the embodiments described above in this document.

[0509] Reference Fig.18 , the encoding device may derive a prediction mode of a current block in a current picture (S1800). The encoding device may derive at least one of the various modes disclosed in this document of the prediction mode.

[0510] The encoding device may generate a predicted luma sample and / or a predicted chroma sample based on the prediction mode (S1810). The encoding device may generate a predicted luma sample by performing prediction on an original sample included in the current block.

[0511] The encoding apparatus may generate prediction-related information based on the prediction mode (S1820). The LMCS-related information may include a bin and / or an LMCS codeword for luma mapping.

[0512] The encoding device may derive bins and LMCS codewords for luma mapping. The encoding device may derive bins and / or LMCS codewords based on SDR or HDR. The encoding device may generate LMCS related information (or LMCS data related information) including bins and / or LCMS codewords for luma mapping.

[0513] The encoding device may generate a mapped predicted luma sample based on a mapping process for the luma sample (S1830). The encoding device may generate a mapped predicted luma sample based on a bin and / or LMCS codeword for luma mapping. For example, the encoding device may derive an input value and a mapping value (output value) of a pivot point for luma mapping, and may generate a mapped predicted luma sample based on the input value and the mapping value. In one example, the encoding device may derive a mapping index idxY based on a first predicted luma sample, and may generate a preliminary mapped predicted luma sample based on the input value and the mapping value of the pivot point corresponding to the mapping index. In another example, a linear mapping (linear shaping, linear LMCS) may be used, and a mapped predicted luma sample may be generated in the linear mapping based on a forward mapping scaling factor derived from two pivot points, and therefore, the index derivation process may be omitted due to the linear mapping.

[0514] The encoding apparatus may generate a residual luma sample based on the mapped predicted luma sample (S1840). For example, the encoding apparatus may derive the residual luma sample based on a difference between the mapped predicted luma sample and the original luma sample.

[0515] The encoding device may derive a chroma residual scaling factor based on the residual luma sample (S1850). The chroma residual scaling factor derivation process may be used for luma-dependent chroma residual scaling. The encoding device may generate a reconstructed luma sample based on the mapped predicted luma sample. In detail, the encoding device may sum the residual luma sample with the mapped predicted luma sample and may generate the reconstructed luma sample based on the summation result. For example, the encoding device may derive a chroma residual scaling coefficient for a plurality of bins based on an LMCS codeword. The encoding device may derive an inverse scaling (bin) index (or a piecewise function index, such as invYIdx) based on the reconstructed luma sample. The encoding device may derive the chroma residual scaling coefficient indicated by the inverse scaling index as a chroma residual scaling factor.

[0516] The encoding device may derive residual chroma samples based on the predicted chroma samples (S1860). The encoding device may derive residual chroma samples based on the original chroma samples and the predicted chroma samples of the current block. For example, the encoding device may derive residual chroma samples based on the difference between the predicted chroma samples and the original chroma samples.

[0517] The encoding device may generate a modified reconstructed luma sample based on a reverse mapping process for the luma sample. The encoding device may generate a modified reconstructed luma sample based on a bin for luma mapping, an LMCS codeword, and a reconstructed luma sample. The encoding device may generate a modified reconstructed luma sample by a reverse mapping process for the reconstructed luma sample. For example, the encoding device may derive a reverse mapping index (e.g., invYIdx) based on the reconstructed luma sample and / or a mapping value (e.g., LmcsPivot[i], i=lmcs_min_bin_idx...LmcsMaxBinIdx+1) specified for the bin index during the reverse mapping process. The encoding device may generate a modified reconstructed luma sample based on the mapping value LmcsPivot[invYIdx] specified for the reverse mapping index.

[0518] The encoding device may generate scaled residual chroma samples (S1870). In detail, the encoding device may generate scaled residual chroma samples based on the residual chroma samples and / or the chroma residual scaling factor. For example, the encoding device may generate scaled residual chroma samples based on the product of the residual chroma samples and the chroma residual scaling factor. In this case, the chroma residual scaling at the encoding end may be referred to as forward chroma residual scaling. Therefore, the chroma residual scaling factor derived by the encoding device may be referred to as a forward chroma residual scaling factor, and a forward scaled residual chroma sample may be generated.

[0519] The encoding device may generate residual information (S1880). In one example, the encoding device may generate residual information based on the mapped predicted luminance samples and the modified reconstructed luminance samples. For example, the encoding device may derive residual information based on scaled residual chrominance samples and / or residual luminance samples. The encoding device may derive transform coefficients based on a transform process of scaled residual chrominance samples and residual luminance samples. For example, the transform process may include at least one of DCT, DST, GBT, or CNT. The encoding device may derive quantized transform coefficients based on a quantization process of the transform coefficients. The quantized transform coefficients may have a one-dimensional vector form based on a coefficient scanning order. The encoding device may generate residual information showing the quantized transform coefficients. The residual information may be generated by various encoding methods such as exponential Golomb, CAVLC, and CABAC.

[0520] The encoding device may encode the image / video information (S1890). The image information may include prediction related information and / or residual information. For example, the image information may also include LMCS related information. For example, the LMCS related information may include linear LMCS related information. For example, at least one LMCS codeword may be derived based on the linear LMCS related information. The encoded video / image information may be output in the form of a bit stream. The bit stream may be sent to the decoding device via a network of storage media.

[0521] The image / video information may include various information items according to the embodiments of this document. For example, the image / video information may include the information disclosed in at least any one of Tables 1 to 56 above.

[0522] In one embodiment, the image information may include information about the absolute value of the chroma residual scaling delta value (e.g., cu_crs_delta_abs) and information about the sign of the chroma residual scaling delta value (e.g., cu_crs_delta_sign_flag). For example, the information about the absolute value of the chroma residual scaling delta value and the information about the sign of the chroma residual scaling delta value may be information about the chroma residual scaling factor. That is, the information about the absolute value of the chroma residual scaling delta value and the information about the sign of the chroma residual scaling delta value may show the chroma residual scaling factor.

[0523] In one embodiment, the encoding device may derive an LMCS codeword. The encoding device may generate LMCS related information based on the LMCS codeword. For example, the chroma residual scaling coefficients of multiple bins may be derived based on the LMCS codeword. The chroma residual scaling factor may be derived based on the chroma residual scaling coefficients of multiple bins.

[0524] In one embodiment, the image information may include an SPS. The SPS may include a first LMCS enable flag indicating whether the LMCS data field is enabled. The LMCS data field may include LMCS related information. That is, the LMCS data field may include information about multiple bins and / or information about LMCS related APSs (e.g., LMCS APS ID).

[0525] In one embodiment, the image information may include header information. In this case, the header information may be information about a slice header or a picture header. Based on the fact that the value of the first LMCS enable flag is 1, the header information may include a second LMCS enable flag showing whether the LMCS data field in the picture is enabled and LMCS-related APS ID information showing the ID of the LMCS APS referenced by the current picture.

[0526] In one embodiment, based on the fact that the value of the second LMCS enable flag is 1, LMCS-related APS ID information may be included in the header information.

[0527] In one embodiment, during the mapping process of luma samples, a mapping index may be derived based on the predicted luma samples, and the mapped predicted luma samples may be generated using a first mapping value based on the mapping index.

[0528] In one embodiment, the LMCS related information may include information about bins used for reverse mapping. A minimum bin index and a maximum bin index may be derived based on the information about the bins. In the reverse mapping process of the luma sample, a reverse mapping index may be derived based on a bin index from the minimum bin index to the maximum bin index based on a mapping value, and a modified reconstructed luma sample may be generated using a second mapping value based on the reverse mapping index.

[0529] In one embodiment, the encoding device may generate a segment index for chroma residual scaling. The encoding device may derive a chroma residual scaling factor based on the segment index. The encoding device may generate scaled residual chroma samples based on the residual chroma samples and the chroma residual scaling factor.

[0530] In one embodiment, when the current block is a single tree architecture or a dual tree architecture (when the current block is a separate tree architecture and the current block is encoded in a separate tree), a chroma residual scaling enable flag showing whether chroma residual scaling is applied to the current block may be generated by the encoding device. Alternatively, regardless of the block tree architecture of the current block, a chroma residual scaling enable flag showing whether chroma residual scaling is applied to the current block may be generated (the chroma residual scaling enable flag may be included in the header information). When chroma residual scaling is applied to the current picture, the current slice and / or the current block, the value of the chroma residual scaling enable flag may be 1.

[0531] In one embodiment, the chroma residual scaling factor may be a single chroma residual scaling factor.

[0532] In one embodiment, the image information may include an SPS (Sequence Parameter Set). The SPS may include a linear LMCS enable flag showing whether the linear LMCS is enabled.

[0533] In one embodiment, the minimum bin index (e.g., lmcs_min_bin_idx) and / or the maximum bin index (e.g., LmcsMaxBinIdx) may be derived based on information about the LMCS data. The first mapping value LmcsPivot[lmcs_min_bin_idx] may be derived based on the minimum bin index. The second mapping value LmcsPivot[LmcsMaxBinIdx] or LmcsPivot[LmcsMaxBinIdx+1] may be derived based on the maximum bin index. The value of the reconstructed luma sample (e.g., lumaSample in Table 36 or Table 37) may be in the range of the first mapping value to the second mapping value. In one embodiment, the values ​​of all reconstructed luma samples may be in the range of the first mapping value to the second mapping value. In another embodiment, the values ​​of some samples of the reconstructed luma samples may be in the range of the first mapping value to the second mapping value.

[0534] In one embodiment, the information about LMCS data may include information about the LMCS data field and the linear LMCS. The information about the linear LMCS may be referred to as information about linear mapping. The LMCS data field may include a linear LMCS flag indicating whether the linear LMCS is applied. When the value of the linear LMCS flag is 1, a mapped predicted luminance sample may be generated based on the information about the linear LMCS.

[0535] In one embodiment, information about the linear LMCS may include information about the first pivot point (e.g., Fig.11 P1 in ) and information about the second pivot point (e.g., Fig.11 For example, the input value and the mapping value of the first pivot point may be the minimum input value and the minimum mapping value, respectively. The input value and the mapping value of the second pivot point may be the maximum input value and the maximum mapping value, respectively. The input values ​​between the minimum input value and the maximum input value may be linearly mapped.

[0536] In one embodiment, the image information may include information about a maximum input value and information about a maximum mapping value. The maximum input value may be the same as the value of the information about the maximum input value (e.g., lmcs_max_input in Table 33). The maximum mapping value may be the same as the value of the information about the maximum mapping value (e.g., lmcs_max_mapped in Table 33).

[0537] In one embodiment, the information about the linear mapping may include information about the input delta value of the second pivot point (e.g., lmcs_max_input_delta in Table 35) and information about the mapped delta value of the second pivot point (e.g., lmcs_max_mapped_delta in Table 35). The maximum input value may be derived based on the input delta value of the second pivot point, and the maximum mapped value may be derived based on the mapped delta value of the second pivot point.

[0538] In one embodiment, the maximum input value and the maximum mapping value may be derived based on at least one formula included in the above-mentioned Table 36.

[0539] In one embodiment, the step of generating the mapped predicted luma samples may include the steps of deriving a forward mapping scaling factor (e.g., ScaleCoeffSingle described above) for the predicted luma samples and generating the mapped predicted luma samples based on the forward mapping scaling factor. The forward mapping scaling factor may be a single factor for predicting luma samples.

[0540] In one embodiment, the forward mapping scaling factor may be derived based on at least one formula included in Table 36 and / or Table 38 above.

[0541] In one embodiment, the mapped predicted luma samples may be derived based on at least one equation included in Table 38 above.

[0542] In one embodiment, the encoding device may derive an inverse mapping scaling factor (e.g., the above-mentioned InvScaleCoeffSingle) of the reconstructed luma sample (e.g., the above-mentioned lumaSample). In addition, the encoding device may generate a modified reconstructed luma sample (e.g., invSample) based on the reconstructed sample and the inverse mapping scaling factor. The inverse mapping scaling factor may be a single factor for reconstructing the luma sample.

[0543] In one embodiment, the inverse mapping scaling factor may be derived using a segment index derived based on the reconstructed luma samples.

[0544] In one embodiment, the segment index may be derived based on the above-mentioned table 50. That is, the comparison process lumaSample included in the table 50 may be repeatedly performed from when the segment index is the minimum bin index to when the segment index is the maximum bin index. <LmcsPivot[idxYInv+1]。

[0545] In one embodiment, the reverse mapping scaling factor may be derived based on at least one equation included in the above-mentioned Tables 33, 34, 35 and 36, or Equation 11 or Equation 12.

[0546] In one embodiment, the modified reconstructed luma samples may be derived based on Equation 20, Equation 21, Table 39 and / or Table 40.

[0547] In one embodiment, the LMCS related information may include information about the number of bins used to derive the mapped predicted luma samples (e.g., lmcs_num_bins_minus1 of Table 41). For example, the number of pivot points used for luma mapping may be set to be the same as the number of bins. In one example, the encoding device may generate incremental input values ​​and incremental mapping values ​​of the pivot points according to the number of bins, respectively. In one example, the input value and mapping value of the pivot point are derived based on the incremental input value (e.g., lmcs_delta_input_cw[i] of Table 41) and the incremental mapping value (e.g., lmcs_delta_mapped_cw[i] of Table 41), and the mapped predicted luma samples may be generated based on the input value (e.g., LmcsPivot_input[i] of Table 42) and the mapping value (e.g., LmcsPivot_mapped[i] of Table 42).

[0548] In one embodiment, the encoding device may derive an LMCS delta codeword based on at least one LMCS codeword and the original codeword OrgCW included in the LMCS-related information, and may derive the mapped luminance prediction samples based on at least one LMCS codeword and the original codeword. In one example, the information about the linear mapping may include the information about the LMCS delta codeword.

[0549] In one embodiment, at least one LMCS codeword may be derived based on the sum of the LMCS delta codeword and OrgCW. For example, OrgCW may be (1<<BitDepthY) / 16, where BitDepthY may indicate the luminance bit depth. This embodiment may be based on Equation 12.

[0550] In one embodiment, at least one LMCS codeword may be derived based on the sum of the LMCS delta codeword and OrgCW*(lmcs_max_bin_idx - lmcs_min_bin_idx + 1), where lmcs_max_bin_idx and lmcs_min_bin_idx may be the maximum bin index and the minimum bin index, respectively, and OrgCW may be (1<<BitDepthY) / 16. This embodiment may be based on Equation 15 and Equation 16.

[0551] In one embodiment, at least one LMCS codeword may be a multiple of 2.

[0552] In one embodiment, when the luminance bit depth BitDepthY of the reconstructed luminance samples is higher than 10, at least one LMCS codeword may be a multiple of 1<<(BitDepthY - 10).

[0553] In one embodiment, at least one LMCS codeword may be in the range of (OrgCW>>1) to (OrgCW<<1)-1.

[0554] In the above paragraphs, the information about the LMCS data may be the same as the LMCS-related information.

[0555] Fig. 20 and Fig.21 Schematically shows an example of an image / video decoding method and related components according to an embodiment of this document. Fig. 20 The method shown may be performed by Figure 3 the decoding device disclosed in. Specifically, for example, in Fig. 20In the above, S2000 may be executed by the entropy decoder 310 of the decoding device, S2010 and S2020 may be executed by the predictor 330 of the decoding device, S2030 may be executed by the residual processor 320 or the predictor 330 of the decoding device, S2040 may be executed by the residual processor 320 of the decoding device, S2050 may be executed by the adder 340 of the decoding device, S2060 may be executed by the residual processor 320, the predictor 330 and / or the adder 340 of the decoding device, and S2070 may be executed by the adder 340 of the decoding device. Fig. 20 The method disclosed in may include the embodiments described above in this document.

[0556] Reference Fig. 20 , the decoding device may receive / obtain video / image information (S2000). The video / image information may include prediction related information and / or residual information. The decoding device may receive / obtain image / video information through a bitstream.

[0557] The image / video information may include various information items according to the embodiments of this document. For example, the image / video information may include the information disclosed in at least any one of Tables 1 to 56 above.

[0558] The decoding apparatus may derive a prediction mode of a current block in a current picture based on the prediction mode information (S2010). The decoding apparatus may derive at least one of various modes disclosed in this document of the inter prediction mode.

[0559] The decoding device may generate a predicted luma sample and a predicted chroma sample (S2020). The decoding device may derive the predicted luma sample and the predicted chroma sample based on the prediction mode. The decoding device may generate the predicted luma / chroma samples by performing prediction on the original samples included in the current block.

[0560] The decoding device may generate a mapped predicted brightness sample (S2030). The decoding device may generate a mapped predicted brightness sample based on a mapping process for the brightness sample. For example, the decoding device may derive an input value and a mapping value (output value) of a pivot point for brightness mapping, and may generate a mapped predicted brightness sample based on the input value and the mapping value. In one example, the decoding device may derive a (forward) mapping index idxY based on the first predicted brightness sample, and may generate a preliminary mapped predicted brightness sample based on the input value and the mapping value of the pivot point corresponding to the mapping index. In another example, a linear mapping (linear shaping, linear LMCS) may be used, and a mapped predicted brightness sample may be generated in the linear mapping based on a forward mapping scaling factor derived from two pivot points, and therefore, the index derivation process may be omitted due to the linear mapping.

[0561] The decoding device may generate residual luma samples based on the residual information (S2040). For example, the decoding device may derive quantized transform coefficients based on the residual information. The quantized transform coefficients may have a one-dimensional vector form based on a coefficient scanning order. The decoding device may derive the transform coefficients based on a dequantization process of the quantized transform coefficients. The decoding device may derive residual samples based on a dequantization process of the transform coefficients. The residual samples may include residual luma samples and / or residual chroma samples.

[0562] The decoding device may generate a reconstructed luma sample (S2050). The decoding device may generate the reconstructed luma sample based on the mapped predicted luma sample. In detail, the decoding device may sum the residual luma sample with the mapped predicted luma sample, and may generate the reconstructed luma sample based on the summation result.

[0563] The decoding device may generate a modified reconstructed luma sample. The decoding device may generate the modified reconstructed luma sample based on an inverse mapping process of the luma sample. The decoding device may generate the modified reconstructed luma sample based on the LMCS related information and the reconstructed luma sample. The decoding device may generate the modified reconstructed luma sample by an inverse mapping process of the reconstructed luma sample.

[0564] The decoding device may derive a chroma residual scaling factor based on the reconstructed luma sample (S2060). The decoding device may derive a chroma residual scaling factor for luma-dependent chroma residual scaling based on the LMCS codeword and the luma sample. For example, the decoding device may derive chroma residual scaling coefficients for multiple bins (including ChromaScaleCoeff[i] in Table 19 and / or Table 20). The decoding device may derive a reverse scaling (bin) index (or piecewise function index) based on the reconstructed luma sample. The decoding device may derive a chroma residual scaling factor based on the chroma residual scaling coefficients of multiple bins and the reverse scaling index. In one example, the decoding device derives the chroma residual scaling coefficient indicated by the reverse scaling bin index among the chroma residual scaling coefficients of multiple bins as the chroma residual scaling factor. The derivation of the reverse scaling index may be implemented based on Table 51 and / or Table 52 above.

[0565] The decoding device may generate a reconstructed chroma sample (S2070). The decoding device may generate the reconstructed chroma sample based on the predicted chroma sample, the residual chroma sample, and the chroma residual scaling factor. For example, the decoding device may derive the product of the chroma residual scaling factor and the residual chroma sample, and may generate the reconstructed chroma sample based on the multiplied result and the predicted chroma sample. The decoding device may generate the reconstructed chroma sample by summing the multiplied result and the predicted chroma sample.

[0566] In one embodiment, the image information may include information about the absolute value of the chroma residual scaling delta value (e.g., cu_crs_delta_abs) and information about the sign of the chroma residual scaling delta value (e.g., cu_crs_delta_sign_flag). For example, the information about the absolute value of the chroma residual scaling delta value and the information about the sign of the chroma residual scaling delta value may be information about the chroma residual scaling factor. That is, the chroma residual scaling factor may be derived based on the information about the absolute value of the chroma residual scaling delta value and the information about the sign of the chroma residual scaling delta value. The scaled residual chroma sample may be generated based on the residual chroma sample and the chroma residual scaling factor. In this case, the chroma residual scaling at the decoding end may be referred to as reverse chroma residual scaling, as opposed to the encoding end. Therefore, the chroma residual scaling factor derived by the decoding device may be referred to as a reverse chroma residual scaling factor, and a reversely scaled residual chroma sample may be generated. The reconstructed chroma sample may be generated based on the predicted chroma sample and the scaled residual chroma sample.

[0567] In one embodiment, the image information may include LMCS related information. For example, the LMCS related information may include the above-mentioned information about luminance mapping (e.g., forward mapping, reverse mapping, and linear mapping), information about chrominance residual scaling and / or indexes related to LMCS (or shaping or shaping device) (e.g., maximum bin index, minimum bin index, and mapping index), information about the absolute value of the chrominance residual scaling increment value, information about the sign of the chrominance residual scaling increment value, and / or information about the number of blocks sharing the same chrominance residual scaling index. The LMCS codeword may be derived based on the LMCS related information. The chrominance residual scaling coefficients of multiple bins may be derived based on the LMCS codeword. The (bin) index for reverse scaling may be derived based on the reconstructed luminance sample. The chrominance residual scaling factor may be derived based on the chrominance residual scaling coefficients of multiple bins and the reverse scaling index.

[0568] In one example, a chroma residual scaling coefficient indicated by an inverse scaling index among the chroma residual scaling coefficients of the plurality of bins may be derived as a chroma residual scaling factor.

[0569] In one embodiment, the inverse scaling index may be derived based on at least two average values ​​of reconstructed luma samples of the current picture.

[0570] In one embodiment, the image information may include an APS. For example, the APS may include an LMCS data field. The LMCS data field may include LMCS related information.

[0571] In one embodiment, the image information may include an SPS. The SPS may include a first LMCS enable flag showing whether the LMCS data field is enabled.

[0572] In one embodiment, the image information may include header information. In this case, the header information may be information about a slice header or a picture header. Based on the fact that the value of the first LMCS enable flag is 1, the header information may include a second LMCS enable flag showing whether the LMCS data field in the picture is enabled.

[0573] In one embodiment, based on the fact that the value of the second LMCS enable flag is 1, LMCS-related APS ID information may be included in the header information.

[0574] In one embodiment, when the current block is a single tree architecture or a dual tree architecture (when the current block is a separate tree architecture and the current block is encoded in a separate tree), a chroma residual scaling enable flag showing whether chroma residual scaling is applied to the current block may be signaled. Alternatively, regardless of the block tree architecture of the current block, a chroma residual scaling enable flag showing whether chroma residual scaling is applied to the current block may be included in the header information (which may be signaled). When chroma residual scaling is applied to the current picture, the current slice and / or the current block, the value of the chroma residual scaling enable flag may be 1.

[0575] In one embodiment, when the values ​​of the first LMCS enable flag and the second LMCS enable flag are 1, the chroma residual scaling enable flag may be included in the header information.

[0576] In one embodiment, during the mapping process of luma samples, a mapping index may be derived based on the predicted luma samples, and the mapped predicted luma samples may be generated using a first mapping value based on the mapping index.

[0577] In one embodiment, the LMCS related information may include information about bins used for reverse mapping. A minimum bin index and a maximum bin index may be derived based on the information about the bins. In the reverse mapping process of the luma samples, a reverse mapping index may be derived based on a bin index from the minimum bin index to the maximum bin index based on a mapping value, and a corrected reconstructed luma sample may be generated using a second mapping value based on the reverse mapping index.

[0578] In one embodiment, the minimum bin index (e.g., lmcs_min_bin_idx) and / or the maximum bin index (e.g., LmcsMaxBinIdx) may be derived based on information about the LMCS data. The first mapping value LmcsPivot[lmcs_min_bin_idx] may be derived based on the minimum bin index. The second mapping value LmcsPivot[LmcsMaxBinIdx] or LmcsPivot[LmcsMaxBinIdx+1] may be derived based on the maximum bin index. The value of the reconstructed luma sample (e.g., lumaSample in Table 51 or Table 52) may be in the range of the first mapping value to the second mapping value. In one embodiment, the values ​​of all reconstructed luma samples may be in the range of the first mapping value to the second mapping value. In another embodiment, the values ​​of some samples of the reconstructed luma samples may be in the range of the first mapping value to the second mapping value.

[0579] In one embodiment, the image information may include an SPS (Sequence Parameter Set). The SPS may include a linear LMCS enable flag showing whether the linear LMCS is enabled.

[0580] In one embodiment, the chroma residual scaling factor may be a single chroma residual scaling factor.

[0581] In one embodiment, the information about LMCS data may include information about the LMCS data field and the linear LMCS. The information about the linear LMCS may be referred to as information about linear mapping. The LMCS data field may include a linear LMCS flag indicating whether the linear LMCS is applied. When the value of the linear LMCS flag is 1, a mapped predicted luminance sample may be generated based on the information about the linear LMCS.

[0582] In one embodiment, information about the linear LMCS may include information about the first pivot point (e.g., Fig.11 P1 in ) and information about the second pivot point (e.g., Fig.11 For example, the input value and the mapping value of the first pivot point may be the minimum input value and the minimum mapping value, respectively. The input value and the mapping value of the second pivot point may be the maximum input value and the maximum mapping value, respectively. The input values ​​between the minimum input value and the maximum input value may be linearly mapped.

[0583] In an embodiment, the image information may include information about a maximum input value and information about a maximum mapping value. The maximum input value may be the same as the value of the information about the maximum input value (e.g., lmcs_max_input in Table 33). The maximum mapping value may be the same as the value of the information about the maximum mapping value (e.g., lmcs_max_mapped in Table 33).

[0584] In an embodiment, the information about the linear mapping may include information about the input delta value of the second pivot point (e.g., lmcs_max_input_delta of Table 35) and information about the mapped delta value of the second pivot point (e.g., lmcs_max_mapped_delta of Table 35). The maximum input value may be derived based on the input delta value of the second pivot point, and the maximum mapped value may be derived based on the mapped delta value of the second pivot point.

[0585] In an embodiment, the maximum input value and the maximum mapping value may be derived based on at least one formula included in the above-mentioned Table 36.

[0586] In one embodiment, generating the mapped predicted luma samples comprises: deriving a forward mapped scaling factor (ie, ScaleCoeffSingle) for the predicted luma samples; and generating the mapped predicted luma samples based on the forward mapped scaling factor. The forward mapped scaling factor may be a single factor for the predicted luma samples.

[0587] In one embodiment, the inverse mapping scaling factor may be derived using a segment index derived based on the reconstructed luma samples.

[0588] In one embodiment, the segment index may be derived based on the above-mentioned Table 51. That is, the comparison process (lumaSample <LmcsPivot[idxYInv+1])。

[0589] In one embodiment, the forward mapping scaling factor may be derived based on at least one formula included in Table 36 and / or Table 38 above.

[0590] In one embodiment, the mapped predicted luma samples may be derived based on at least one equation included in Table 38 above.

[0591] In one embodiment, the decoding device may derive an inverse mapping scaling factor (i.e., InvScaleCoeffSingle) of the reconstructed luma sample (i.e., lumaSample). Additionally, the decoding device may generate a modified reconstructed luma sample (i.e., invSample) based on the reconstructed luma sample and the inverse mapping scaling factor. The inverse mapping scaling factor may be a single factor of the reconstructed luma sample.

[0592] In one embodiment, the reverse mapping scaling factor may be derived based on at least one equation included in the above-mentioned Table 33, Table 34, Table 35 and Table 36, or Equation 11 or Equation 12.

[0593] In one embodiment, the modified reconstructed luminance samples may be derived based on Equation 20, Equation 21, Table 39, and / or Table 40 described above.

[0594] In one embodiment, the LMCS-related information may include information on the number of bins for the predicted luminance samples used to derive the mapping (i.e., lmcs_num_bins_minus1 in Table 41). For example, the number of pivot points of the luminance mapping may be set to be equal to the number of bins. In one example, the decoding device may generate the incremental input values and incremental mapping values of the pivot points according to the number of bins, respectively. In one example, the input values and mapping values of the pivot points are derived based on the incremental input values (i.e., lmcs_delta_input_cw[i] in Table 41) and the incremental mapping values (i.e., lmcs_delta_mapped_cw[i] in Table 41), and the mapped predicted luminance samples may be generated based on the input values (i.e., LmcsPivot_input[i] in Table 42) and the mapping values (i.e., LmcsPivot_mapped[i] in Table 42).

[0595] In one embodiment, the decoding device may derive the LMCS incremental codeword based on at least one LMCS codeword and the original codeword (OrgCW) included in the LMCS-related information, and may derive the mapped luminance prediction samples based on at least one LMCS codeword and the original codeword. In one example, the information on the linear mapping may include information on the LMCS incremental codeword.

[0596] In one embodiment, at least one LMCS codeword may be derived based on the sum of the LMCS incremental codeword and OrgCW. For example, OrgCW is (1<<BitDepthY) / 16, where BitDepthY represents the luminance bit depth. This embodiment may be based on Equation 14.

[0597] In one embodiment, at least one LMCS codeword may be derived based on the sum of the LMCS incremental codeword and OrgCW*(lmcs_max_bin_idx - lmcs_min_bin_idx + 1). For example, lmcs_max_bin_idx and lmcs_min_bin_idx are the maximum bin index and the minimum bin index, respectively, and OrgCW may be (1<<BitDepthY) / 16. This embodiment may be based on Equation 15 and Equation 16.

[0598] In one embodiment, at least one LMCS codeword may be a multiple of 2.

[0599] In one embodiment, when the luma bit depth (BitDepthY) of the reconstructed luma samples is higher than 10, at least one LMCS codeword may be a multiple of 1<<(BitDepthY-10).

[0600] In one embodiment, at least one LMCS codeword may be in the range from (OrgCW>>1) to (OrgCW<<1)-1.

[0601] In the above-mentioned embodiment, the method is described based on a flow chart with a series of steps or boxes. The present disclosure is not limited to the order of the above-mentioned steps or boxes. Some steps or boxes may occur simultaneously or in an order different from other steps or boxes as described above. In addition, it will be understood by those skilled in the art that the steps shown in the above-mentioned flow chart are not exclusive, and may include additional steps, or one or more steps in the flow chart may be deleted without affecting the scope of the present disclosure.

[0602] The method according to the above-mentioned embodiment of the present document can be implemented in the form of software, and the encoding device and / or decoding device according to the present document can be included in a device for performing image processing of a TV, computer, smart phone, set-top box, display device, etc.

[0603] When the embodiments in this document are implemented in software, the above methods can be implemented as modules (processes, functions, etc.) that perform the above functions. The modules can be stored in a memory and executed by a processor. The memory can be located inside or outside the processor and can be connected to the processor by various well-known means. The processor may include an application-specific integrated circuit (ASIC), other chipsets, logic circuits, and / or data processing devices. The memory may include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium, and / or other storage devices. That is, the embodiments described in this document can be implemented and executed on a processor, a microprocessor, a controller, or a chip. For example, the functional units shown in the various figures can be implemented and executed on a computer, a processor, a microprocessor, a controller, or a chip. In this case, information about the instructions or algorithms used to implement can be stored in a digital storage medium.

[0604] In addition, the decoding device and the encoding device to which the present disclosure is applied may be included in a multimedia broadcast sending / receiving device, a mobile communication terminal, a home theater video device, a digital theater video device, a surveillance camera, a video chat device, a real-time communication device (e.g., video communication), a mobile streaming device, a storage medium, a camera, a VoD service providing device, an over-the-top (OTT) video device, an Internet streaming service providing device, a three-dimensional (3D) video device, a teleconference video device, a transportation user device (i.e., a vehicle user device, an aircraft user device, a ship user device, etc.), and a medical video device, and may be used to process video signals and data signals. For example, an over-the-top (OTT) video device may include a game console, a Blu-ray player, an Internet access TV, a home theater system, a smart phone, a tablet PC, a digital video recorder (DVR), etc.

[0605] In addition, the processing method to which this document is applied can be generated in the form of a program to be executed by a computer, and can be stored in a computer-readable recording medium. Multimedia data with a data structure according to the present disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices that store data readable by a computer system. For example, a computer-readable recording medium may include a BD, a universal serial bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. In addition, a computer-readable recording medium includes a medium implemented in the form of a carrier wave (i.e., transmission via the Internet). In addition, the bit stream generated by the encoding method can be stored in a computer-readable recording medium or can be sent via a wired / wireless communication network.

[0606] In addition, the embodiments of the present document may be implemented as a computer program product according to a program code, and the program code may be executed in a computer through the embodiments of the present document. The program code may be stored on a computer-readable carrier.

[0607] Fig. 22 An example of a content streaming system to which the embodiments disclosed in this document can be applied is shown.

[0608] Reference Fig. 22 , a content streaming system to which the implementation methods of this document are applied may mainly include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[0609] The encoding server compresses the content input from the multimedia input device (e.g., a smart phone, a camera, a camcorder, etc.) into digital data to generate a bitstream and sends the bitstream to the streaming server. As another example, when the multimedia input device (e.g., a smart phone, a camera, a camcorder, etc.) directly generates the bitstream, the encoding server may be omitted.

[0610] A bitstream may be generated by an encoding method or a bitstream generating method to which an embodiment of the present disclosure is applied, and a streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0611] The streaming server sends multimedia data to the user device based on the user's request through the network server, and the network server is used as a medium to inform the user of the service. When the user requests the desired service from the network server, the network server transmits it to the streaming server, and the streaming server sends the multimedia data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server is used to control the command / response between the devices in the content streaming system.

[0612] The streaming server may receive content from a media storage device and / or an encoding server. For example, when receiving content from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bitstream for a predetermined time.

[0613] Examples of user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation, tablet computers, tablet PCs, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays), digital TVs, desktop computers, digital signage, etc. Each server in the content streaming system may operate as a distributed server, in which case data received from each server may be distributed.

[0614] Each server in the content streaming system may operate as a distributed server, and in this case, data received from each server may be distributed and processed.

[0615] The claims described herein may be combined in various ways. For example, the technical features of the method claims of this document may be combined and implemented as a device, and the technical features of the device claims of this document may be combined and implemented as a method. In addition, the technical features of the method claims of this document and the technical features of the device claims may be combined to be implemented as a device, and the technical features of the method claims of this document and the technical features of the device claims may be combined and implemented as a method.

Claims

1. An image decoding method performed by a decoding device, the image decoding method comprising the following steps: Obtaining image information including prediction related information and residual information from a bitstream; deriving a prediction mode for a current block in a current picture based on the prediction related information; generating predicted luma samples and predicted chroma samples based on the prediction mode; generating mapped predicted luma samples based on the mapping process of the predicted luma samples; generating residual luma samples and residual chroma samples based on the residual information; generating reconstructed luma samples based on the mapped predicted luma samples and the residual luma samples; deriving a chroma residual scaling factor based on the reconstructed luma samples; as well as generating reconstructed chroma samples based on the predicted chroma samples, the residual chroma samples and the chroma residual scaling factor, The image information includes information about an absolute value of a chroma residual scaling increment value and information about a sign of the chroma residual scaling increment value. wherein a reverse scaling index is derived based on the reconstructed luminance samples, wherein the chroma residual scaling factor is derived based on the reverse scaling index, the information about the absolute value of the chroma residual scaling delta value, and the information about the sign of the chroma residual scaling delta value, wherein scaled residual chroma samples are generated based on the residual chroma samples and the chroma residual scaling factor, and The reconstructed chroma samples are generated based on the predicted chroma samples and the scaled residual chroma samples.

2. An image encoding method performed by an encoding device, the image encoding method comprising the following steps: deriving a prediction mode for a current block of a current picture; generating predicted luma samples and predicted chroma samples based on the prediction mode; generating prediction related information based on the prediction mode; generating mapped predicted luma samples based on the mapping process of the predicted luma samples; generating residual luma samples based on the mapped predicted luma samples; deriving a chroma residual scaling factor based on the residual luma samples; generating residual chroma samples based on the predicted chroma samples; generating scaled residual chroma samples based on the residual chroma samples and the chroma residual scaling factor; generating residual information based on the residual luma samples and the scaled residual chroma samples; as well as encoding the image information including the prediction related information and the residual information, The image information includes information about an absolute value of a chroma residual scaling increment value and information about a sign of the chroma residual scaling increment value. wherein a reverse scaling index is derived based on the mapped predicted luma samples and the residual luma samples, and The reverse scaling index, the information about the absolute value of the chroma residual scaling increment value, and the information about the sign of the chroma residual scaling increment value are used to derive the chroma residual scaling factor.

3. A method for storing a bit stream, the method storing a bit stream generated by an encoding method in a non-transitory computer-readable storage medium, wherein: The encoding method comprises the following steps: deriving a prediction mode for a current block of a current picture; generating predicted luma samples and predicted chroma samples based on the prediction mode; generating prediction related information based on the prediction mode; generating mapped predicted luma samples based on the mapping process of the predicted luma samples; generating residual luma samples based on the mapped predicted luma samples; deriving a chroma residual scaling factor based on the residual luma samples; generating residual chroma samples based on the predicted chroma samples; generating scaled residual chroma samples based on the residual chroma samples and the chroma residual scaling factor; generating residual information based on the residual luma samples and the scaled residual chroma samples; and encoding the image information including the prediction related information and the residual information to output the bit stream, The image information includes information about an absolute value of a chroma residual scaling increment value and information about a sign of the chroma residual scaling increment value. wherein a reverse scaling index is derived based on the mapped predicted luma samples and the residual luma samples, and The reverse scaling index, the information about the absolute value of the chroma residual scaling increment value, and the information about the sign of the chroma residual scaling increment value are used to derive the chroma residual scaling factor.

4. A method for transmitting data for image information, the method comprising the following steps: Obtaining a bitstream of the image information, wherein the bitstream is generated based on the following steps: deriving a prediction mode for a current block of a current picture, generating a predicted luminance sample and a predicted chrominance sample based on the prediction mode, generating prediction related information based on the prediction mode, generating a mapped predicted luminance sample based on a mapping process of the predicted luminance sample, generating a residual luminance sample based on the mapped predicted luminance sample, deriving a chrominance residual scaling factor based on the residual luminance sample, generating a residual chrominance sample based on the predicted chrominance sample, generating a scaled residual chrominance sample based on the residual chrominance sample and the chrominance residual scaling factor, generating residual information based on the residual luminance sample and the scaled residual chrominance sample, and encoding the image information including the prediction related information and the residual information; and sending said data comprising said bit stream, The image information includes information about an absolute value of a chroma residual scaling increment value and information about a sign of the chroma residual scaling increment value. wherein a reverse scaling index is derived based on the mapped predicted luma samples and the residual luma samples, and The reverse scaling index, the information about the absolute value of the chroma residual scaling increment value, and the information about the sign of the chroma residual scaling increment value are used to derive the chroma residual scaling factor.

Images