Video or image coding based on luma mapping and chroma scaling.

Abstract

To provide a method and apparatus for increasing video / image coding efficiency.SOLUTION: According to the disclosure of the present document, a luma mapping with chroma scaling (LMCS) process may be simplified, and a linear LMCS may be applied. Accordingly, the complexity of the LMCS may be reduced and video / image coding efficiency may be increased.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This technology relates to video or image coding based on chroma mapping and chroma scaling. [Background technology]

[0002] In recent years, the demand for high-resolution, high-quality images and videos, such as 4K or 8K or higher UHD (Ultra High Definition) images and videos, has been increasing in various fields. As image and video data become higher resolution and higher quality, the amount of information or bits transmitted relative to existing image and video data increases. Therefore, when transmitting image data using existing wired or wireless broadband lines, or storing image and video data using existing storage media, transmission and storage costs increase.

[0003] Furthermore, in recent years, interest in and demand for immersive media such as VR (Virtual Reality), AR (Artificial Reality), and holograms have increased, and there has been a rise in broadcasting of images / videos with image characteristics different from real-world images, such as game images.

[0004] Therefore, highly efficient image / video compression technology is required to effectively compress, transmit, store, and play back high-resolution, high-quality image / video information with the various characteristics described above.

[0005] Furthermore, the LMCS (luma mapping with chroma scaling) procedure is used to improve compression efficiency and enhance subjective / objective visual quality, and there are discussions on reducing the computational complexity of the LMCS procedure. [Overview of the project] [Problems that the invention aims to solve]

[0006] According to one embodiment of this document, a method and apparatus for improving image / video coding efficiency are provided. [Means for solving the problem]

[0007] According to one embodiment of this document, an efficient filtering method and apparatus are provided.

[0008] According to one embodiment of this document, an efficient method and apparatus for applying LMCS are provided.

[0009] According to one embodiment of this document, the LMCS codeword (or its range) can be restricted.

[0010] According to one embodiment of this document, a single chroma register dual scaling factor directly signaled in the chroma scaling of LMCS may be used.

[0011] According to one embodiment of this document, a linear mapping (linear LMCS) may be used.

[0012] According to one embodiment of this document, information regarding pivot points necessary for linear mapping can be explicitly signaled.

[0013] According to one embodiment of this document, a flexible number of bins can be used for luma mapping.

[0014] According to one embodiment of this document, the procedure for deriving an inverse mapping index for a Luma sample can be simplified.

[0015] According to one embodiment of this document, a video / image decoding method performed by a decoding device is provided.

[0016] According to one embodiment of this document, a decoding device for video / image decoding is provided.

[0017] According to one embodiment of this document, a video / image encoding method performed by an encoding device is provided.

[0018] According to one embodiment of this document, an encoding device for performing video / image encoding is provided.

[0019] According to one embodiment of this document, a computer-readable digital storage medium storing encoded video / image information generated by the video / image encoding method disclosed in at least one of the embodiments of this document is provided.

[0020] According to one embodiment of this document, a computer-readable digital storage medium storing encoded information or encoded video / image information that causes a decoding device to perform the video / image decoding method disclosed in at least one of the embodiments of this document is provided.

Advantages of the Invention

[0021] According to one embodiment of this document, the overall image / video compression efficiency can be increased.

[0022] According to one embodiment of this document, the subjective / objective visual quality can be enhanced through efficient filtering.

[0023] According to one embodiment of this document, the LMCS procedure for image / video coding can be efficiently executed.

[0024] According to one embodiment of this document, the resources / costs (software or hardware) required for the LMCS procedure can be minimized.

[0025] According to one embodiment of this document, the hardware implementation for the LMCS procedure can be facilitated.

[0026] According to one embodiment of this document, the division operations required to derive the LMCS codeword in mapping (reshaping) can be eliminated or minimized through the LMCS codeword (or range thereof) constraint.

[0027] According to one embodiment of this document, latency due to piece-wise index identification can be eliminated by using a single chroma regi dual scaling factor.

[0028] According to one embodiment of this paper, the ChromaRegi dual scaling procedure can be performed in LMCS without relying on the restoration of Chroma Blocks via the use of linear mapping, and thus latency in scaling can be eliminated.

[0029] According to one embodiment of this document, mapping efficiency in LMCS can be improved.

[0030] According to one embodiment of this paper, the complexity of LMCS can be reduced by simplifying the procedure for deriving inverse mapping indices or inverse scaling indices for chroma regi dual scaling for luma samples, and thus the video / image coding efficiency can be increased. [Brief explanation of the drawing]

[0031] [Figure 1] An example of a video / image coding system to which the embodiments described herein may be applied is schematically shown. [Figure 2] This figure schematically illustrates the configuration of a video / image encoding device to which the embodiments described herein may be applied. [Figure 3] This figure schematically illustrates the configuration of a video / image decoding device to which the embodiments described herein may be applied. [Figure 4] This shows an example of a video / image encoding method based on interpretation. [Figure 5]This example shows a video / image decoding method based on interpretation. [Figure 6] An example of an interpretation prediction procedure is shown below. [Figure 7] This illustrates the hierarchical structure for coded images / videos. [Figure 8] This document illustrates the hierarchical structure of CVS according to one embodiment. [Figure 9] An exemplary LMCS structure according to one embodiment of this document is shown. [Figure 10] The LMCS structure according to another embodiment described in this document is shown. [Figure 11] A graph illustrating an exemplary forward mapping is shown. [Figure 12] This flowchart illustrates a method for deriving a Chromaregi dual-scaling index according to one embodiment of this document. [Figure 13] This document shows a linear fitting of the pivot point according to one embodiment. [Figure 14] An example of a linear reshaper according to one embodiment of this document is shown. [Figure 15] This document presents an example of linear forward mapping in one embodiment. [Figure 16] This document presents an example of inverse forward mapping in one embodiment. [Figure 17] An example of a video / image encoding method and related components according to the embodiments (etc.) of this document is schematically shown. [Figure 18] An example of a video / image encoding method and related components according to the embodiments (etc.) of this document is schematically shown. [Figure 19] An example of an image / video decoding method and related components according to the embodiments described herein is outlined. [Figure 20] An example of an image / video decoding method and related components according to the embodiments described herein is outlined. [Figure 21] Examples of content streaming systems to which embodiments disclosed in this document may be applied are shown. [Modes for carrying out the invention]

[0032] The disclosures in this document can be modified in various ways and may have various embodiments, but specific embodiments are illustrated in the drawings and described in detail. However, this does not mean that the disclosure is intended to limit itself to any particular embodiment. The terms used in this document are used solely to describe specific embodiments and are not intended to limit the technical ideas of the embodiments described herein. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this document, terms such as “includes” or “has” are intended to indicate the presence of features, figures, steps, actions, components, parts, or combinations thereof described in the document, and should be understood not to preemptively exclude the possibility of the presence or addition of one or more different features, figures, steps, actions, components, parts, or combinations thereof.

[0033] On the other hand, the configurations shown in the drawings described in this document are shown independently for the purpose of explaining their distinct characteristic functions, and this does not mean that each configuration is implemented with separate hardware or separate software. For example, two or more configurations may be combined to form one configuration, and one configuration may be divided into multiple configurations. Embodiments in which each configuration is integrated and / or separated are also included in the scope of disclosure of this document.

[0034] The embodiments described in this document will be explained below with reference to the attached diagrams. Hereafter, the same reference numerals may be used for the same components in the diagrams, and redundant descriptions of the same components may be omitted.

[0035] Figure 1 schematically shows an example of a video / image coding system to which the embodiments described in this document may be applied.

[0036] As shown in Figure 1, a video / image coding system may comprise a first device (source device) and a second device (receiving device). The source device can transmit encoded video / image information or data to the receiving device in file or streaming form via a digital storage medium or network.

[0037] The source device may comprise a video source, an encoding device, and a transmitter. The receiving device may comprise a receiver, a decoding device, and a renderer. The encoding device may be called a video / image encoding device, and the decoding device may be called a video / image decoding device. The transmitter may be provided in the encoding device. The receiver may be provided in the decoding device. The renderer may comprise a display unit, which may consist of a separate device or external component.

[0038] A video source can acquire video / images through processes such as video / image capture, synthesis, or generation. A video source may include video / image capture devices and / or video / image generation devices. Video / image capture devices may include, for example, one or more cameras, or a video / image archive containing previously captured video / images. Video / image generation devices may include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images may be generated via a computer, in which case the video / image capture process may be replaced by the process of generating the associated data.

[0039] An encoding device can encode input video / images. For compression and coding efficiency, the encoding device can perform a series of steps including prediction, transformation, and quantization. The encoded data (encoded video / image information) can be output in bitstream format.

[0040] The transmitting unit can transmit encoded video / image information or data output in bitstream format to the receiving unit of a receiving device via a digital storage medium or network in file or streaming format. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitting unit may include elements for generating media files via a predetermined file format and may include elements for transmission via a broadcast / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0041] A decoding device can decode video / images by performing a series of steps, such as inverse quantization, inverse transformation, and prediction, corresponding to the operation of an encoding device.

[0042] The renderer can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0043] This document relates to video / image coding. For example, the methods / embodiments disclosed herein can be applied to methods disclosed in the VVC (versatile video coding) standard. Furthermore, the methods / embodiments disclosed herein can be applied to methods disclosed in the EVC (essential video coding) standard, AV1 (AOMedia Video 1) standard, AVS2 (2nd generation of audio video coding standard), or next-generation video / image coding standards (e.g., 267 or H.268).

[0044] This document presents various embodiments relating to video / image coding, and unless otherwise noted, these embodiments may be combined with each other.

[0045] In this document, "video" can mean a collection of images over time. "Picture" generally refers to a unit representing a single image at a specific time point in time, and "slice / tile" is a unit that constitutes part of a picture in coding. A slice / tile can contain one or more CTUs (coding tree units). A single picture can consist of one or more slices / tiles. A tile is a rectangular region of CTUs within a particular tile column and a particular tile row in a picture. The tile column is a rectangular region of CTUs having a height equal to the height of the picture and a width specified by syntax elements in the picture parameter set. The tile row is a rectangular region of CTUs having a height specified by syntax elements in the picture parameter set and a width equal to the width of the picture.A tile scan may represent a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a tile, whereas tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. A slice may include an integer number of complete tiles or an integer number of consecutive complete CTU rows within a tile of a picture that may be exclusively contained in a single NAL unit.

[0046] On the other hand, a single picture can be divided into two or more subpictures. A subpicture can be a rectangular region of one or more slices within a picture.

[0047] A pixel or pel can refer to the smallest unit that makes up a picture (or image). Alternatively, the term "sample" may be used as a counterpart to pixel. A sample can generally represent a pixel or a pixel value, and can represent only the luma component pixel / pixel value, or only the chroma component pixel / pixel value.

[0048] A unit can represent a basic unit of image processing. A unit can contain at least one of a specific region of a picture and information associated with that region. A unit can contain one luma block and two chroma (e.g., cb, cr) blocks. The term unit may sometimes be used interchangeably with terms such as block or area. In general, an M×N block can contain a sample (or sample array) or a set (or array) of transform coefficients consisting of M columns and N rows.

[0049] In this document, "A or B" may mean "just A," "just B," or "both A and B." In other words, in this document, "A or B" may be interpreted as "A and / or B." For example, in this document, "A, B or C" may mean "just A," "just B," "just C," or "any combination of A, B and C."

[0050] The slashes ( / ) and commas used in this document can mean "and / or". For example, "A / B" can mean "A and / or B". Thus, "A / B" can mean "just A", "just B", or "both A and B". For example, "A, B, C" can mean "A, B or C".

[0051] In this document, "at least one of A and B" may mean "just A," "just B," or "both A and B." Furthermore, in this document, the expressions "at least one of A or B" and "at least one of A and / or B" may be interpreted similarly to "at least one of A and B."

[0052] Furthermore, in this document, "at least one of A, B and C" may mean "just A," "just B," "just C," or "any combination of A, B and C." Also, "at least one of A, B or C" or "at least one of A, B and / or C" may mean "at least one of A, B and C."

[0053] Furthermore, parentheses used in this document may mean "for example." Specifically, when "prediction (intra prediction)" is displayed, "intra prediction" may be proposed as an example of "prediction." In other words, "prediction" in this document is not limited to "intra prediction," and "intra prediction" may be proposed as an example of "prediction." Also, when "prediction (i.e., intra prediction)" is displayed, "intra prediction" may be proposed as an example of "prediction."

[0054] Technical features described individually within a single drawing in this document may be implemented individually or simultaneously.

[0055] Figure 2 is a schematic diagram illustrating the configuration of a video / image encoding device to which the embodiments described in this document may be applied. Hereinafter, the term "encoding device" may include an image encoding device and / or a video encoding device.

[0056] As shown in Figure 2, the encoding device 200 can be configured to include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-prediction unit 221 and an intra-prediction unit 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be called a rebuilder or a reconstructed block generator. The image segmentation unit 210, prediction unit 220, residual processing unit 230, entropy encoding unit 240, addition unit 250, and filtering unit 260 described above can be configured by one or more hardware components (e.g., an encoder chipset or processor) depending on the embodiment. The memory 270 may also include a DPB (decoded picture buffer) and may be configured by a digital storage medium. The hardware components may further include the memory 270 as an internal / external component.

[0057] The image splitting unit 210 can split an input image (or picture, frame) input to the encoding device 200 into one or more processing units. For example, the processing units may be called coding units (CUs). In this case, the coding units can be recursively split from a coding tree unit (CTU) or the largest coding unit (LCU) using a QTBTTT (Quad-tree binary-tree ternary-tree) structure. For example, one coding unit can be split into multiple coding units of deeper depth based on a quad-tree structure, a binary-tree structure, and / or a ternary structure. In this case, for example, the quad-tree structure may be applied first, followed by the binary-tree structure and / or the ternary structure. Alternatively, the binary-tree structure may be applied first. The coding procedure according to this disclosure may be performed based on the final coding unit that is not further split. In this case, based on coding efficiency due to image characteristics, the largest coding unit can be used as the final coding unit, or, if necessary, the coding unit can be recursively divided into lower-depth coding units so that the optimally sized coding unit is used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later. As another example, the processing unit may further comprise a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit can each be separated or partitioned from the final coding unit described above.The prediction unit may be a unit of sample prediction, and the conversion unit may be a unit for deriving conversion coefficients and / or a unit for deriving a residual signal from conversion coefficients.

[0058] The term "unit" can sometimes be used interchangeably with terms such as "block" or "area." Generally, an M×N block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only the luminance (luma) component pixel / pixel value, or only the chroma component pixel / pixel value. A sample can be used as the term corresponding to a single picture (or image) pixel or pel.

[0059] The encoding device 200 can generate a residual signal (residual block, residual sample array) by subtracting the prediction signal (predicted block, predicted sample array) output from the inter-prediction unit 221 or intra-prediction unit 222 from the input image signal (original block, original sample array), and the generated residual signal is transmitted to the conversion unit 232. In this case, as shown in the figure, the unit that subtracts the prediction signal (predicted block, predicted sample array) from the input image signal (original block, original sample array) within the encoder 200 can be called the subtraction unit 231. The prediction unit can make predictions for the block to be processed (hereinafter referred to as the current block) and generate a predicted block that includes the predicted sample for the current block. The prediction unit can determine whether intra-prediction or inter-prediction is applied on a current block or CU basis. The prediction unit can generate various prediction-related information, such as prediction mode information, and transmit it to the entropy encoding unit 240, as will be described later in the explanation of each prediction mode. The prediction information can be encoded by the entropy encoding unit 240 and output in bitstream format.

[0060] The intra-prediction unit 222 can predict the current block by referring to a sample in the current picture. The referenced sample can be located adjacent to the current block or at a distance, depending on the prediction mode. The prediction mode in intra-prediction can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC mode and planar mode. Directional modes can include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of prediction direction. However, this is illustrative, and more or fewer directional prediction modes may be used depending on the settings. The intra-prediction unit 222 can also determine the prediction mode to apply to the current block using the prediction modes applied to adjacent blocks.

[0061] The interprediction unit 221 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between adjacent blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, adjacent blocks may include spatially adjacent blocks that exist in the current picture and temporally adjacent blocks that exist in the reference picture. The reference picture containing the reference block and the reference picture containing the temporally adjacent block may be the same or different. The temporally adjacent block may be called a collocated reference block, col CU, etc., and the reference picture containing the temporally adjacent block may be called a collocated picture (colPic). For example, the inter-prediction unit 221 can construct a motion information candidate list based on adjacent blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter-prediction can be performed based on various prediction modes; for example, in skip mode and merge mode, the inter-prediction unit 221 can use the motion information of adjacent blocks as the motion information of the current block. In skip mode, unlike merge mode, a residual signal may not be transmitted.In motion vector prediction (MVP) mode, the motion vector of an adjacent block is used as a motion vector predictor, and the motion vector difference is signaled to indicate the motion vector of the current block.

[0062] The prediction unit 220 can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for predictions on a single block, and can also apply intra-prediction and inter-prediction simultaneously. This can be called combined inter and intra prediction (CIIP). The prediction unit can also base its predictions on an intra-block copy (IBC) prediction mode or a palette mode for predictions on a block. The IBC prediction mode or palette mode can be used for content image / video coding such as in games, for example, as in SCC (screen content coding). IBC basically performs predictions within the current picture, but can be performed similarly to inter-prediction in that it derives reference blocks within the current picture. That is, IBC can use at least one of the inter-prediction techniques described in this document. Palette mode can be considered an example of intra-coding or intra-prediction. When palette mode is applied, sample values ​​within the picture can be signaled based on information about the palette table and palette index.

[0063] The prediction signal generated via the prediction unit (including the inter-prediction unit 221 and / or the intra-prediction unit 222) can be used to generate a reconstructed signal or a residual signal. The transformation unit 232 can apply a transformation technique to the residual signal to generate transformation coefficients. For example, the transformation technique may include at least one of DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT refers to a transformation obtained from a graph when relational information between pixels is represented in a graph. CNT refers to a transformation obtained by generating a prediction signal using all previously reconstructed pixels and obtaining a transformation based on it. The transformation process may also be applied to pixel blocks of the same size and square shape, or to non-square blocks of variable size.

[0064] The quantization unit 233 quantizes the conversion coefficients and transmits them to the entropy encoding unit 240, which can encode the quantized signal (information about the quantized conversion coefficients) and output it as a bitstream. The information about the quantized conversion coefficients can be called residual information. The quantization unit 233 can rearrange the block-form quantized conversion coefficients into a one-dimensional vector form based on the coefficient scan order, and can also generate information about the quantized conversion coefficients based on the one-dimensional vector form of the quantized conversion coefficients. The entropy encoding unit 240 can perform various encoding methods, such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding). In addition to the quantized conversion coefficients, the entropy encoding unit 240 can also encode information necessary for video / image restoration (e.g., the values ​​of syntax elements) together with or separately from the quantized conversion coefficients. Encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream form in units of network abstraction layer (NAL). The video / image information may further include information about various parameter sets, such as adaptation parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). The video / image information may also further include general constraint information. In this document, information and / or syntax elements transmitted / signaled from the encoding device to the decoding device may be included in the video / image information. The video / image information may be encoded via the encoding procedure described above and included in the bitstream.The bitstream can be transmitted over a network or stored in a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The signal output from the entropy encoding unit 240 can be transmitted by a transmitting unit (not shown) and / or stored by a storage unit (not shown) which are configured as internal / external elements of the encoding device 200, or the transmitting unit may be included in the entropy encoding unit 240.

[0065] The quantized conversion coefficients output from the quantization unit 233 can be used to generate a prediction signal. For example, a residual signal (residual block or residual sample) can be reconstructed by applying inverse quantization and inverse transformation to the quantized conversion coefficients via the inverse quantization unit 234 and the inverse transformation unit 235. The adder 155 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter-prediction unit 221 or the intra-prediction unit 222. If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The adder 250 can be called the reconstruction unit or reconstructed block generation unit. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, or, as described later, for inter-prediction of the next picture after filtering.

[0066] On the other hand, LMCS (luma mapping with chroma scaling) can also be applied during the picture encoding and / or restoration process.

[0067] The filtering unit 260 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 260 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 270, specifically in the DPB of the memory 270. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter. The filtering unit 260 can generate various filtering-related information and transmit it to the entropy encoding unit 240, as will be described later in the explanation of each filtering method. The filtering-related information can be encoded by the entropy encoding unit 240 and output in bitstream format.

[0068] The corrected restored picture sent to memory 270 can be used as a reference picture in the interpretation unit 221. When interpretation is applied via this, the encoding device can avoid prediction mismatches between the encoding device 100 and the decoding device, and can also improve encoding efficiency.

[0069] The DPB in memory 270 can store the corrected restored picture for use as a reference picture in the inter-prediction unit 221. Memory 270 can store motion information of blocks from which motion information in the current picture has been derived (or encoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 221 for use as motion information of spatially adjacent blocks or motion information of temporally adjacent blocks. Memory 270 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 222.

[0070] Figure 3 is a schematic diagram illustrating the configuration of a video / image decoding device to which the embodiments described in this document may be applied. Hereinafter, the term "decoding device" may include an image decoding device and / or a video decoding device.

[0071] As shown in Figure 3, the decoding device 300 can be configured to include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an intra-predictor 331 and an inter-predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 321. The entropy decoder 310, residual processor 320, predictor 330, adder 340, and filtering device 350 described above can be configured by a single hardware component (e.g., a decoder chipset or processor) depending on the embodiment. The memory 360 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The aforementioned hardware component may also further include memory 360 as an internal / external component.

[0072] When a bitstream containing video / image information is input, the decoding device 300 can reconstruct the image corresponding to the process by which the video / image information was processed in the encoding device shown in Figure 3. For example, the decoding device 300 can derive units / blocks based on block division-related information obtained from the bitstream. The decoding device 300 can perform decoding using the processing units applied in the encoding device. Therefore, the decoding processing unit can be, for example, a coding unit, which can be divided from a coding tree unit or a maximum coding unit according to a quad-tree structure, a binary tree structure, and / or a terminally tree structure. One or more conversion units can be derived from the coding unit. The reconstructed image signal decoded and output via the decoding device 300 can then be reproduced via a playback device.

[0073] The decoding device 300 can receive the signal output from the encoding device shown in Figure 3 in bitstream form, and the received signal can be decoded via the entropy decoding unit 310. For example, the entropy decoding unit 310 can parse the bitstream to derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information about various parameter sets, such as the adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. The decoding device can further decode the picture based on the parameter set information and / or the general constraint information. The signaling / received information and / or syntax elements described later in this document can be decoded via the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 310 can decode information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements necessary for image reconstruction and the quantized values ​​of conversion coefficients related to the residual. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element in the bitstream, determines a context model using the syntax element information to be decoded and the decoded information of adjacent and decoded blocks or symbol / bin information decoded in a previous step, predicts the probability of bin occurrence based on the determined context model, performs arithmetic decoding of the bins, and generates symbols corresponding to the values ​​of each syntax element. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the context model of the next symbol / bin.Of the information decoded by the entropy decoding unit 310, information related to prediction is provided to the prediction unit (inter-prediction unit 332 and intra-prediction unit 331), and the residual values ​​that have been entropy decoded by the entropy decoding unit 310, i.e., quantized conversion coefficients and related parameter information, can be input to the residual processing unit 320. The residual processing unit 320 can derive residual signals (residual blocks, residual samples, residual sample arrays). In addition, of the information decoded by the entropy decoding unit 310, information related to filtering can be provided to the filtering unit 350. On the other hand, a receiving unit (not shown) that receives signals output from the encoding device can be further configured as an internal / external element of the decoding device 300, or the receiving unit can be a component of the entropy decoding unit 310. On the other hand, the decoding device relating to this document may be called a video / image / picture decoding device, and the decoding device may also be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoding unit 310, and the sample decoder may include at least one of the inverse quantization unit 321, inverse transformation unit 322, addition unit 340, filtering unit 350, memory 360, inter-prediction unit 332, and intra-prediction unit 331.

[0074] The inverse quantization unit 321 can inverse quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 321 can rearrange the quantized transformation coefficients in a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device. The inverse quantization unit 321 can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) and obtain the transformation coefficients.

[0075] In the inverse conversion unit 322, the conversion coefficients are inversely converted to obtain a residual signal (residual block, residual sample array).

[0076] The prediction unit can make predictions for the current block and generate a predicted block containing prediction samples for the current block. Based on the prediction information output from the entropy decoding unit 310, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block, and can determine a specific intra / inter-prediction mode.

[0077] The prediction unit 320 can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for prediction of a single block, and can also apply intra-prediction and inter-prediction simultaneously. This can be called combined inter and intra prediction (CIIP). The prediction unit can also base its prediction on an intra-block copy (IBC) prediction mode or on a palette mode for prediction of a block. The IBC prediction mode or palette mode can be used for content image / video coding such as in games, for example, as in SCC (screen content coding). IBC basically performs prediction within the current picture, but can be performed similarly to inter-prediction in that it derives a reference block within the current picture. That is, IBC can utilize at least one of the inter-prediction techniques described in this document. Palette mode can be considered an example of intra-coding or intra-prediction. When palette mode is applied, information about the palette table and palette index can be included in the video / image information and signaled.

[0078] The intra-prediction unit 331 can predict the current block by referring to a sample in the current picture. The referenced sample can be located adjacent to or far from the current block depending on the prediction mode. In intra-prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. The intra-prediction unit 331 can also determine the prediction mode to be applied to the current block using the prediction modes applied to adjacent blocks.

[0079] The interprediction unit 332 can derive a predicted block for the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in blocks, subblocks, or samples based on the correlation of motion information between adjacent blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, adjacent blocks may include spatially adjacent blocks that exist in the current picture and temporally adjacent blocks that exist in the reference picture. For example, the interprediction unit 332 can construct a motion information candidate list based on adjacent blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction can be performed based on various prediction modes, and the prediction information may include information indicating the mode of interprediction for the current block.

[0080] The summing unit 340 can generate a restored signal (restored picture, restored block, restored sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the prediction unit (which comprises an inter-prediction unit 332 and / or an intra-prediction unit 331). If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the restored block.

[0081] The summing unit 340 may be called the restoration unit or restoration block generation unit. The generated restoration signal can be used for intra-prediction of the next block to be processed in the current picture, and can be output after filtering as described later, or it can be used for intra-prediction of the next picture.

[0082] On the other hand, LMCS (luma mapping with chroma scaling) can also be applied during the picture decoding process.

[0083] The filtering unit 350 can apply filtering to the restored signal to improve subjective / objective image quality. For example, the filtering unit 350 can apply various filtering methods to the restored picture to generate a modified restored picture, and can transmit the modified restored picture to the memory 360, specifically to the DPB of the memory 360. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.

[0084] The (modified) restored picture stored in the DPB of memory 360 can be used as a reference picture by the inter-prediction unit 332. Memory 360 can store motion information of blocks from which motion information in the current picture has been derived (or decoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 260 for use as motion information of spatially adjacent blocks or motion information of temporally adjacent blocks. Memory 360 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 331.

[0085] In this specification, embodiments described for the filtering unit 260, inter-prediction unit 221, and intra-prediction unit 222 of the encoding device 200 can also be applied identically or in a corresponding manner to the filtering unit 350, inter-prediction unit 332, and intra-prediction unit 331 of the decoding device 300, respectively.

[0086] As mentioned above, prediction is performed to improve compression efficiency when performing video coding. Through this, a predicted block containing predicted samples for the current block, which is the block to be coded, can be generated. Here, the predicted block contains predicted samples in the spatial domain (or pixel domain). The predicted block is derived in both the encoding and decoding devices, and the encoding device can improve image coding efficiency by signaling the decoding device with information (residual information) about the residual between the original block and the predicted block, which is not the original sample value of the original block itself. The decoding device can derive a residual block containing residual samples based on the residual information, and can combine the residual block and the predicted block to generate a restored block containing restored samples, and can generate a restored picture containing the restored block.

[0087] The residual information can be generated through transformation and quantization procedures. For example, an encoding device can signal the relevant residual information (via a bitstream) to a decoding device by deriving a residual block between the original block and the predicted block, performing a transformation procedure on the residual samples (residual sample array) contained in the residual block to derive transformation coefficients, and then performing a quantization procedure on the transformation coefficients to derive quantized transformation coefficients. Here, the residual information may include information such as the value information, position information, transformation technique, transformation kernel, and quantization parameters of the quantized transformation coefficients. The decoding device can derive a residual sample (or residual block) by performing an inverse quantization / inverse transformation procedure based on the residual information. The decoding device can generate a reconstructed picture based on the predicted block and the residual block. The encoding device can also derive a residual block by inverse quantization / inverse transformation of the quantized transformation coefficients for reference for subsequent interpretation of the picture, and generate a reconstructed picture based on this.

[0088] In this document, at least one of quantization / inverse quantization and / or transformation / inverse transformation may be omitted. If quantization / inverse quantization is omitted, the quantized transformation coefficients may be called transformation coefficients. If transformation / inverse transformation is omitted, the transformation coefficients may also be called coefficients or residual coefficients, or for consistency of expression, they may still be called transformation coefficients.

[0089] In this document, quantized transformation coefficients and transformation coefficients may be referred to as transformation coefficients and scaled transformation coefficients, respectively. In this case, residual information may include information about the transformation coefficients, which may be signaled via residual coding syntax. Transformation coefficients may be derived based on the residual information (or information about the transformation coefficients), and scaled transformation coefficients may be derived via an inverse transformation (scaling) of the transformation coefficients. Residual samples may be derived based on an inverse transformation (transformation) of the scaled transformation coefficients. This may be applied / expressed similarly in other parts of this document.

[0090] Intra prediction can represent a prediction that generates prediction samples for the current block based on reference samples within the picture to which the current block belongs (hereinafter referred to as the current picture). When intra prediction is applied to the current block, adjacent reference samples to be used for intra prediction of the current block can be derived. The adjacent reference samples of the current block may include samples adjacent to the left boundary of the current block of size nW × nH and a total of 2 × nH samples adjacent to the bottom left, samples adjacent to the top boundary of the current block and a total of 2 × nW samples adjacent to the top right, and one sample adjacent to the top left of the current block. Alternatively, the adjacent reference samples of the current block may include multiple columns of upper adjacent samples and multiple rows of left adjacent samples. Furthermore, the adjacent reference samples of the current block may also include a total of nH samples adjacent to the right boundary of the current block, which is of size nW × nH, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the bottom-right of the current block.

[0091] However, some of the adjacent reference samples in the current block may not yet be decoded or available. In this case, the decoder can construct adjacent reference samples to use for prediction by substituting the unavailable samples as available samples, or by constructing adjacent reference samples to use for prediction through interpolation of available samples.

[0092] If neighboring reference samples are derived, (i) predicted samples can be derived based on the average or interpolation of neighboring reference samples in the current block, or (ii) predicted samples can be derived based on neighboring reference samples in the current block that are located in a specific (predicted) direction relative to the predicted sample. Case (i) may be called non-directional mode or non-angular mode, and case (ii) may be called directional mode or angular mode.

[0093] Furthermore, the predicted sample can also be generated by interpolation between a first adjacent sample located in the prediction direction of the current block's intra-prediction mode and a second adjacent sample located in the opposite direction of the prediction direction, using the predicted sample of the current block from the adjacent reference samples as a reference. In the above case, it can be called linear interpolation intra-prediction (LIP). Alternatively, a linear model can be used to generate chroma predicted samples based on chroma samples. In this case, it can be called LM mode.

[0094] Alternatively, temporary predicted samples for the current block can be derived based on filtered adjacent reference samples, and the predicted samples for the current block can be derived by performing a weighted sum on the temporary predicted samples and at least one reference sample derived by the intra-prediction mode from the existing adjacent reference samples, i.e., unfiltered adjacent reference samples. In the above case, it can be called PDPC (Position dependent intra-prediction).

[0095] Furthermore, intra-predictive coding can be performed by selecting the reference sample line with the highest prediction accuracy from among the adjacent multi-reference sample lines in the current block, deriving a predicted sample using the reference sample located in the prediction direction on that line, and then instructing (signaling) the decoding device with the reference sample line used at this time. In the case described above, this can be called multi-reference line intra-prediction or MRL-based intra-prediction.

[0096] Furthermore, the current block can be divided into vertical or horizontal subpartitions, and intra-prediction can be performed based on the same intra-prediction mode. Adjacent reference samples can then be derived and used on a subpartition-by-subpartition basis. In other words, in this case, the intra-prediction mode for the current block is also applied to the subpartition, and by deriving and using adjacent reference samples on a subpartition-by-subpartition basis, intra-prediction performance can be improved in some cases. Such a prediction method can be called ISP (intra sub-partitions) based intra-prediction.

[0097] The intra-prediction method described above can be called an intra-prediction type, distinct from the intra-prediction mode. The intra-prediction type can be referred to by various terms, such as intra-prediction technique or additional intra-prediction mode. For example, the intra-prediction type (or additional intra-prediction mode, etc.) can include at least one of the aforementioned LIP, PDPC, MRL, and ISP. A general intra-prediction method that excludes specific intra-prediction types such as LIP, PDPC, MRL, and ISP can be called a normal intra-prediction type. The normal intra-prediction type can be applied generally when the aforementioned specific intra-prediction types are not applicable, and predictions can be performed based on the aforementioned intra-prediction modes. On the other hand, post-processing filtering can be performed on the derived prediction samples as needed.

[0098] Specifically, the intra-prediction procedure may include an intra-prediction mode / type determination step, an adjacent reference sample derivation step, and an intra-prediction mode / type-based predictive sample derivation step. Additionally, a post-filtering step may be performed on the derived predictive samples as needed.

[0099] When intraprediction is applied, the intraprediction mode applied to the current block can be determined by utilizing the intraprediction modes of adjacent blocks. For example, the decoder can select one of the MPM candidates in the MPM (most probable mode) list derived based on the intraprediction modes of the current block's adjacent blocks (e.g., left and / or upper adjacent blocks) and additional candidate modes, based on the received MPM index, or it can select one of the remaining intraprediction modes not included in the MPM candidates (and planar modes), based on the remaining intraprediction mode information. The MPM list may or may not include planar modes as candidates. For example, if the MPM list includes planar modes as candidates, it may have six candidates; if the MPM list does not include planar modes as candidates, it may have five candidates. If the MPM list does not include planar mode as a candidate, a not-planar flag (e.g., intra_luma_not_planar_flag) can be signaled to indicate that the current intra-prediction mode of the block is not planar mode. For example, the MPM flag may be signaled first, and the MPM index and not-planar flag may be signaled if the value of the MPM flag is 1. The MPM index may also be signaled if the value of the not-planar flag is 1. Here, the reason why the MPM list is configured not to include planar mode as a candidate is not because the planar mode is not an MPM, but because planar mode is always considered as an MPM, so the flag (not-planar flag) is signaled first to check whether it is planar mode or not.

[0100] For example, whether the intra-prediction mode applied to the current block is in the MPM candidate (and planar mode) or in the remaining mode can be indicated based on the MPM flag (e.g., intra_luma_mpm_flag). A value of 1 for the MPM flag indicates that the intra-prediction mode for the current block is in the MPM candidate (and planar mode), and a value of 0 for the MPM flag indicates that the intra-prediction mode for the current block is not in the MPM candidate (and planar mode). A value of 0 for the not-planar flag (e.g., intra_luma_not_planar_flag) indicates that the intra-prediction mode for the current block is planar mode, and a value of 1 for the not-planar flag indicates that the intra-prediction mode for the current block is not planar mode. The MPM index can be signaled in the form of an mpm_idx or intra_luma_mpm_idx syntex element, and the remaining intra-prediction mode information can be signaled in the form of a rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntex element. For example, the remaining intra-prediction mode information can be indexed in order of prediction mode number among the overall intra-prediction modes that are not included in the MPM candidate (and planar mode) and point to one of them. The intra-prediction mode is an intra-prediction mode for a luma component (sample). The intra prediction mode information may include at least one of the following: the MPM flag (e.g., intra_luma_mpm_flag), the not planar flag (e.g., intra_luma_not_planar_flag), the MPM index (e.g., mpm_idx or intra_luma_mpm_idx), or the remaining intra prediction mode information (rem_intra_luma_pred_mode or intra_luma_mpm_remainder).In this document, the MPM list may be referred to by various terms, such as the MPM candidate list, candModeList, etc. When an MIP is applied to the current block, a separate mpm flag (e.g., intra_mip_mpm_flag), an mpm index (e.g., intra_mip_mpm_idx), and remaining intra predictive mode information (e.g., intra_mip_mpm_remainder) for the MIP may be signaled, while the not planar flag is not signaled.

[0101] In other words, when an image is generally divided into blocks, the current block to be coded and its neighboring blocks will have similar image characteristics. Therefore, the current block and its neighboring blocks have a high probability of having the same or similar intra-prediction modes. Thus, the encoder can utilize the intra-prediction mode of the neighboring block to encode the intra-prediction mode of the current block.

[0102] For example, an encoder / decoder can configure an MPM (most probable modes) list for the current block. This MPM list can also be referred to as an MPM candidate list. Here, MPM can mean a mode used to improve coding efficiency by considering the similarity between the current block and adjacent blocks during intra predictive mode coding. As mentioned above, the MPM list can be configured to include planar modes or to exclude planar modes. For example, if the MPM list includes planar modes, the number of candidates in the MPM list is 6. If the MPM list does not include planar modes, the number of candidates in the MPM list is 5.

[0103] The encoder / decoder can configure an MPM list containing five or six MPMs.

[0104] To construct an MPM list, three types of modes can be considered: default intra modes, neighbor intra modes, and derived intra modes.

[0105] For the adjacent intra-mode described above, two adjacent blocks, namely the left adjacent block and the upper adjacent block, can be considered.

[0106] As mentioned above, if the MPM list is configured not to include planar mode, planar mode is excluded from the list, and the number of MPM list candidates can be set to 5.

[0107] Furthermore, among the intra-prediction modes, non-directional modes (or non-angle modes) may include average-based DC modes or interpolation-based planar modes of the neighboring reference samples of the current block.

[0108] When interpretation is applied, the prediction unit of the encoding / decoding device can perform interpretation on a block-by-block basis to derive predicted samples. Interpretation can indicate predictions derived in a manner dependent on data elements (e.g., sample values ​​or motion information) of pictures other than the current picture. When interpretation is applied to the current block, a predicted block (predicted sample array) for the current block can be derived based on the reference block (reference sample array) identified by the motion vector on the reference picture pointed to by the index of the reference picture. In this case, in order to reduce the amount of motion information transmitted in interpretation mode, the motion information of the current block can be predicted on a block, subblock, or sample basis based on the correlation of motion information between neighboring blocks and the current block. The motion information may include the motion vector and the index of the reference picture. The motion information may further include information on the interpretation type (L0 prediction, L1 prediction, Bi prediction, etc.). When interpretation is applied, neighboring blocks may include spatial neighboring blocks that exist within the current picture and temporal neighboring blocks that exist in the reference picture. The reference picture containing the aforementioned reference block and the reference picture containing the aforementioned temporally adjacent block may be the same or different. The temporally adjacent block may be referred to by names such as collocated reference block, colCU, etc., and the reference picture containing the aforementioned temporally adjacent block may be referred to as collocated picture (colPic). For example, a candidate list of motion information can be constructed based on the adjacent blocks of the current block, and flags or index information can be signaled to indicate which candidate is selected (used) in order to derive the motion vector and / or index of the reference picture of the current block.Interpretation is performed based on various prediction modes. For example, in skip mode and merge mode, the motion information of the current block may be identical to the motion information of the selected adjacent block. In skip mode, unlike merge mode, a residual signal may not be transmitted. In motion vector prediction (MVP) mode, the motion vector of the selected adjacent block can be used as a motion vector predictor, and the motion vector difference can be signaled. In this case, the motion vector of the current block can be derived using the sum of the motion vector predictor and the motion vector difference.

[0109] The motion information may include L0 motion information and / or L1 motion information depending on the interpretation type (L0 prediction, L1 prediction, Bi prediction, etc.). A motion vector in the L0 direction may be called an L0 motion vector or MVL0, and a motion vector in the L1 direction may be called an L1 motion vector or MVL1. A prediction based on an L0 motion vector may be called an L0 prediction, a prediction based on an L1 motion vector may be called an L1 prediction, and a prediction based on both the L0 motion vector and the L1 motion vector may be called a bi (Bi) prediction. Here, an L0 motion vector may represent a motion vector associated with a reference picture list L0 (L0), and an L1 motion vector may represent a motion vector associated with a reference picture list L1 (L1). The reference picture list L0 may include pictures earlier in the output order than the current picture, and the reference picture list L1 may include pictures later in the output order than the current picture. The aforementioned earlier picture may be called a forward (reference) picture, and the aforementioned later picture may be called a reverse (reference) picture. The reference picture list L0 may include further reference pictures that are later in the output order than the current picture. In this case, the earlier picture may be indexed first in the reference picture list L0, and the later picture may be indexed afterward. The reference picture list L1 may include further reference pictures that are earlier in the output order than the current picture. In this case, the later picture may be indexed first in the reference picture list L1, and the earlier picture may be indexed afterward. Here, the output order may correspond to the POC (picture order count) order.

[0110] Figure 4 shows an example of a video / image encoding method based on interpretation.

[0111] The encoding device performs interpretation for the current block (S400). The encoding device derives the interpretation mode and motion information of the current block and can generate predicted samples for the current block. Here, the interpretation mode determination, motion information derivation, and predicted sample generation procedures can be performed simultaneously, and any one procedure can be performed before the others. For example, the interpretation unit of the encoding device may include a prediction mode determination unit, a motion information derivation unit, and a predicted sample derivation unit. The prediction mode determination unit can determine the prediction mode for the current block, the motion information derivation unit can derive the motion information of the current block, and the predicted sample derivation unit can derive predicted samples for the current block. For example, the interpretation unit of the encoding device can search for blocks similar to the current block within a certain area (search area) of the reference picture via motion estimation and derive reference blocks whose difference from the current block is the minimum or below a certain standard. Based on this, a reference picture index pointing to the reference picture in which the reference block is located can be derived, and a motion vector can be derived based on the positional difference between the reference block and the current block. The encoding device can determine which of the various prediction modes is applied to the current block. The encoding device can compare the RD costs for the various prediction modes and determine the optimal prediction mode for the current block.

[0112] For example, when skip mode or merge mode is applied to the current block, the encoding device can configure a merge candidate list, as described later, and derive a reference block from among the reference blocks pointed to by the merge candidates included in the merge candidate list whose difference from the current block is the minimum or below a certain standard. In this case, a merge candidate associated with the derived reference block is selected, and merge index information pointing to the selected merge candidate is generated and signaled to the decoding device. The movement information of the current block can be derived using the movement information of the selected merge candidate.

[0113] As another example, when the (A)MVP mode is applied to the current block, the encoding device can configure the (A)MVP candidate list described later, and use the motion vector of the selected mvp candidate from among the mvp (motion vector predictor) candidates included in the (A)MVP candidate list as the mvp of the current block. In this case, for example, the motion vector pointing to the reference block derived by the motion estimation described above can be used as the motion vector of the current block, and the mvp candidate having the motion vector with the smallest difference from the motion vector of the current block can become the selected mvp candidate. The MVD (motion vector difference), which is the difference obtained by subtracting the mvp from the motion vector of the current block, can be derived. In this case, information regarding the MVD can be signaled to the decoding device. Also, when the (A)MVP mode is applied, the value of the reference picture index can be composed of reference picture index information and separately signaled to the decoding device.

[0114] The encoding device can derive a residual sample based on the predicted sample (S410). The encoding device can derive the residual sample by comparing the original sample of the current block with the predicted sample.

[0115] The encoding device encodes image information including prediction information and residual information (S420). The encoding device can output the encoded image information in bitstream format. The prediction information is information related to the prediction procedure and may include prediction mode information (e.g., skip flag, merge flag, or mode index) and motion information. The motion information may include candidate selection information (e.g., merge index, mvp flag, or mvp index) which is information for deriving a motion vector. The motion information may also include the above-mentioned MVD information and / or reference picture index information. Furthermore, the motion information may include information indicating whether L0 prediction, L1 prediction, or bi prediction is applied. The residual information is information about the residual sample. The residual information may include information about the quantized conversion coefficients for the residual sample.

[0116] The output bitstream can be stored in a (digital) storage medium and transmitted to a decoding device, or it can be transmitted to a decoding device via a network.

[0117] On the other hand, as described above, the encoding device can generate a reconstructed picture (including a reconstructed sample and a reconstructed block) based on the reference sample and the residual sample. This is because the encoding device can derive the same prediction results as the decoding device, thereby increasing coding efficiency. Therefore, the encoding device can store the reconstructed picture (or reconstructed sample, reconstructed block) in memory and use it as a reference picture for interpretation. As described above, in-loop filtering procedures and the like can be further applied to the reconstructed picture.

[0118] A video / image decoding procedure based on interpretation may, in general, include the following:

[0119] Figure 5 shows an example of a video / image decoding method based on interpretation.

[0120] As shown in Figure 5, the decoding device can perform operations corresponding to those performed by the encoding device. The decoding device can make predictions for the current block based on the received prediction information and derive prediction samples.

[0121] Specifically, the decoding device can determine the prediction mode for the current block based on the received prediction information (S500). The decoding device can determine which inter-prediction mode is applied to the current block based on the prediction mode information in the prediction information.

[0122] For example, based on the merge flag, it can be determined whether the merge mode is applied to the current block or whether the (A)MVP mode is determined. Alternatively, one of several inter-prediction mode candidates can be selected based on the mode index. The inter-prediction mode candidates may include skip mode, merge mode, and / or (A)MVP mode, or may include various inter-prediction modes as described later.

[0123] The decoding device derives motion information for the current block based on the determined interpretation mode (S510). For example, if a skip mode or merge mode is applied to the current block, the decoding device can configure a merge candidate list, as described later, and select one merge candidate from among the merge candidates included in the merge candidate list. This selection can be made based on the selection information (merge index) described above. The motion information for the current block can be used to derive motion information for the selected merge candidate. The motion information for the selected merge candidate can be used as motion information for the current block.

[0124] As another example, when the (A)MVP mode is applied to the current block, the decoding device can configure the (A)MVP candidate list described later, and use the motion vector of the selected mvp candidate from among the mvp (motion vector predictor) candidates included in the (A)MVP candidate list as the mvp of the current block. The selection can be made based on the selection information (mvp flag or mvp index) described above. In this case, the MVD of the current block can be derived based on the information regarding the MVD, and the motion vector of the current block can be derived based on the mvp of the current block and the MVD. Furthermore, the reference picture index of the current block can be derived based on the reference picture index information. In the reference picture list for the current block, the picture pointed to by the reference picture index can be derived as the reference picture referenced for interpretation of the current block.

[0125] On the other hand, as will be described later, the movement information of the current block can be derived without constructing a candidate list, and in this case, the movement information of the current block can be derived according to the procedure disclosed in the prediction mode described later. In this case, the candidate list configuration described above can be omitted.

[0126] The decoding device can generate predicted samples for the current block based on the motion information of the current block (S520). In this case, the reference picture can be derived based on the reference picture index of the current block, and the predicted samples for the current block can be derived using the sample of the reference block pointed to by the motion vector of the current block on the reference picture. In this case, as will be described later, a further predicted sample filtering procedure may be performed on all or some of the predicted samples for the current block.

[0127] For example, the interpretation unit of the decoding device may include a prediction mode determination unit, a motion information derivation unit, and a prediction sample derivation unit. The prediction mode determination unit determines the prediction mode for the current block based on the prediction mode information received, the motion information derivation unit derives motion information (motion vector and / or reference picture index, etc.) for the current block based on the motion information received, and the prediction sample derivation unit derives the prediction sample for the current block.

[0128] The decoding device generates a residual sample for the current block based on the received residual information (S530). The decoding device generates a restored sample for the current block based on the predicted sample and the residual sample, and can generate a restored picture based on this (S540). As described above, in-loop filtering procedures and the like may be further applied to the restored picture thereafter.

[0129] Figure 6 illustrates the interpretation prediction procedure.

[0130] Referring to Figure 6, as described above, the interpretation procedure may include an interpretation mode determination step, a motion information derivation step based on the determined prediction mode, and a prediction execution (prediction sample generation) step based on the derived motion information. The interpretation procedure may be performed by an encoding device and a decoding device, as described above. In this document, the coding device may include an encoding device and / or a decoding device.

[0131] As shown in Figure 6, the coding device determines the interpretation mode for the current block (S600). Various interpretation modes can be used to predict the current block in the picture. For example, various modes can be used, such as merge mode, skip mode, MVP (motion vector prediction) mode, affine mode, subblock merge mode, and MMVD (merge with MVD) mode. DMVR (Decoder side motion vector refinement) mode, AMVR (adaptive motion vector resolution) mode, Bi-prediction with CU-level weight (BCW), and Bi-directional optical flow (BDOF) can be used as additional or alternative modes. The affine mode can also be called the affine motion prediction mode. The MVP mode can also be called the AMVP (advanced motion vector prediction) mode. In this document, motion information candidates derived by some modes and / or some modes may also be included as one of the motion information related candidates of other modes. For example, an HMVP candidate can be added as a merge candidate in the merge / skip mode, or as an MVP candidate in the MVP mode. When the HMVP candidate is used as a motion information candidate in the merge mode or skip mode, the HMVP candidate can be called an HMVP merge candidate.

[0132] Prediction mode information indicating the inter-prediction mode of the current block can be signaled from the encoding device to the decoding device. The prediction mode information can be included in the bitstream and received by the decoding device. The prediction mode information may include index information indicating one of several candidate modes. Alternatively, the inter-prediction mode can be indicated via hierarchical signaling of flag information. In this case, the prediction mode information may include one or more flags. For example, a skip flag may be signaled to indicate whether a skip mode is applicable, and if the skip mode is not applicable, a merge flag may be signaled to indicate whether a merge mode is applicable, and if the merge mode is not applicable, it may indicate that the MVP mode is applicable, or flags for additional distinctions may be further signaled. Affine modes can be signaled as independent modes, or as modes dependent on merge modes or MVP modes, etc. For example, affine modes may include affine merge mode and affine MVP mode.

[0133] On the other hand, the current block may be signaled with information indicating whether the list0(L0) prediction, list1(L1) prediction, or bi-prediction described above is used in the current block (current coding unit). This information can be called motion prediction direction information, inter-prediction direction information, or inter-prediction instruction information, and can be composed / encoded / signaled, for example, in the form of an inter_pred_idc syntax element. That is, the inter_pred_idc syntax element can indicate whether the list0(L0) prediction, list1(L1) prediction, or bi-prediction described above is used in the current block (current coding unit). For the sake of explanation, in this document, the inter-prediction type (L0 prediction, L1 prediction, or BI prediction) pointed to by the inter_pred_idc syntax element can be expressed as motion prediction direction. L0 prediction can also be expressed as pred_L0, L1 prediction as pred_L1, and bi-prediction as pred_BI. For example, depending on the value of the inter_pred_idc syntax element, the following prediction types can be represented:

[0134] [Table 1]

[0135] As described above, a single picture can contain one or more slices. A slice can have one of the following slice types: I (intra) slices, P (predictive) slices, and B (bi-predictive) slices. The slice type can be indicated based on the slice type information. For blocks in an I slice, only intra prediction can be used for prediction, and inter-predictive prediction is not used. Of course, in this case as well, it is possible to code and signal the original sample values ​​without prediction. For blocks in a P slice, intra or inter-predictive prediction may be used, and if inter-predictive prediction is used, only uni prediction may be used. On the other hand, for blocks in a B slice, intra or inter-predictive prediction may be used, and if inter-predictive prediction is used, up to bi-predictive prediction may be used.

[0136] L0 and L1 can contain reference pictures that were encoded / decoded before the current picture. For example, L0 can contain reference pictures that are earlier and / or later than the current picture in the POC order, and L1 can contain reference pictures that are later and / or earlier than the current picture in the POC order. In this case, L0 may be assigned a lower reference picture index relative to the reference pictures that are earlier than the current picture in the POC order, and L1 may be assigned a lower reference picture index relative to the reference pictures that are later than the current picture in the POC order. In the case of a B slice, biprediction can be applied, and in this case, unidirectional biprediction or bidirectional biprediction can be applied. Bidirectional biprediction can be called true biprediction.

[0137] The coding device derives motion information for the current block (S610). The motion information can be derived based on the interprediction mode.

[0138] The coding device can perform interpretation using the motion information of the current block. The encoding device can derive optimal motion information for the current block through a motion estimation procedure. For example, the encoding device can use the original block in the original picture for the current block to search for a highly correlated similar reference block in fractional pixel units within a defined search range in the reference picture, thereby deriving motion information. Block similarity can be derived based on the difference in sample values ​​based on phase. For example, block similarity can be calculated based on the SAD between the current block (or the template of the current block) and the reference block (or the template of the reference block). In this case, motion information can be derived based on the reference block with the smallest SAD within the search area. The derived motion information can be signaled to the decoding device in various ways based on the interpretation mode.

[0139] The coding device performs inter prediction based on motion information for the current block (S620). The coding device can derive predicted samples (etc.) for the current block based on the motion information. The current block containing the predicted samples can be called a predicted block.

[0140] When merge mode is applied, the movement information of the current predicted block is not transmitted directly, but rather the movement information of the surrounding predicted blocks is used to derive the movement information of the current predicted block. Therefore, the movement information of the current predicted block can be specified by transmitting flag information indicating that merge mode has been used and a merge index indicating which surrounding predicted block was used. This merge mode can be called regular merge mode.

[0141] To perform a merge, the encoder must search for merge candidate blocks to be used to derive motion information for the currently predicted block. For example, up to five merge candidate blocks may be used, but the embodiments of this document are not limited to this. The maximum number of merge candidate blocks can be transmitted in the slice header or tile group header. After searching for the merge candidate blocks, the encoder can generate a merge candidate list and select the merge candidate block with the lowest cost from among them as the final merge candidate block.

[0142] The merge candidate list may, for example, use five merge candidate blocks. For example, it may use four spatial merge candidates and one temporal merge candidate. Hereinafter, the spatial merge candidate, or the spatial MVP candidate described later, may be called an SMVP, and the temporal merge candidate, or the temporal MVP candidate described later, may be called a TMVP.

[0143] Figure 7 illustrates the hierarchical structure for coded images / videos.

[0144] Referring to Figure 7, coded images / videos are divided into the VCL (video coding layer), which handles the decoding process and the images / videos themselves; a lower-level system that transmits and stores the coded information; and the NAL (network abstraction layer), which exists between the VCL and the lower-level system and is responsible for network adaptation functions.

[0145] VCL can generate VCL data containing compressed image data (slice data), or generate parameter sets containing information such as Picture Parameter Set (PPS), Sequence Parameter Set (SPS), and Video Parameter Set (VPS), or SEI (Supplemental Enhancement Information) messages additionally required during the image decoding process.

[0146] In NAL, a NAL unit can be generated by adding header information (NAL unit header) to the RBSP (Raw Byte Sequence Payload) generated by VCL. In this case, the RBSP refers to the slice data, parameter set, SEI message, etc., generated by VCL. The NAL unit header can include NAL unit type information, which is identified by the RBSP data contained in the NAL unit.

[0147] As shown in the above diagram, NAL units can be divided into VCL NAL units and Non-VCL NAL units by the RBSP generated in VCL. A VCL NAL unit can mean a NAL unit that contains information about the image (slice data), and a Non-VCL NAL unit can mean a NAL unit that contains information necessary to decode the image (parameter set or SEI message).

[0148] The aforementioned VCL NAL units and Non-VCL NAL units can be transmitted over a network with header information added according to the data standards of the lower-level system. For example, NAL units can be transformed into data formats of predetermined standards such as H.266 / VVC file format, RTP (Real-time Transport Protocol), and TS (Transport Stream) and transmitted over various networks.

[0149] As mentioned above, the NAL unit type can be identified by the RBSP data structure contained within the NAL unit, and information about such NAL unit types can be stored in the NAL unit header and signaled.

[0150] For example, NAL units can be broadly classified into VCL NAL unit types and Non-VCL NAL unit types depending on whether or not they contain information (slice data) about the image. VCL NAL unit types can be further classified by the nature and type of picture they contain, while Non-VCL NAL unit types can be further classified by the type of parameter set.

[0151] The following is an example of a NAL unit type identified by the type of parameter set included in the Non-VCL NAL unit type.

[0152] -APS (Adaptation Parameter Set) NAL unit: Type for NAL units that include APS

[0153] -DPS (Decoding Parameter Set) NAL unit: Type for NAL units including DPS

[0154] -VPS (Video Parameter Set) NAL unit: Type for NAL unit including VPS

[0155] -SPS (Sequence Parameter Set) NAL unit: Type for NAL units that include SPS

[0156] -PPS (Picture Parameter Set) NAL unit: Type for NAL units that include PPS

[0157] -PH (Picture header) NAL unit: Type for NAL units that include PH

[0158] The aforementioned NAL unit type has syntax information for the NAL unit type, and this syntax information can be stored in the NAL unit header and signaled. For example, the syntax information is nal_unit_type, and the NAL unit type can be identified by the nal_unit_type value.

[0159] On the other hand, as mentioned above, a single picture can contain multiple slices, and a single slice can contain a slice header and slice data. In this case, a single picture header may be added to each of the multiple slices (slice headers and slice data sets) within a single picture. The picture header (picture header syntax) may contain information / parameters that can be commonly applied to the picture. In this document, slices may be mixed with or replaced by tile groups. Also in this document, slice headers may be mixed with or replaced by type group headers.

[0160] The slice header (slice header syntax) may include information / parameters that can be commonly applied to the slice. The APS (APS syntax) or PPS (PPS syntax) may include information / parameters that can be commonly applied to one or more slices or pictures. The SPS (SPS syntax) may include information / parameters that can be commonly applied to one or more sequences. The VPS (VPS syntax) may include information / parameters that can be commonly applied to multiple layers. The DPS (DPS syntax) may include information / parameters that can be commonly applied to video in general. The DPS may include information / parameters related to the concatenation of CVS (coded video sequence). In this document, High-level syntax (HLS) may include at least one of the APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, picture header syntax, and slice header syntax.

[0161] In this document, the image / video information encoded from the encoding device to the decoding device and signaled in bitstream form may include not only partitioning-related information, intra / inter prediction information, residual information, and in-loop filtering information within the picture, but also information contained in the slice header, the picture header, the APS, the PPS, the SPS, the VPS, and / or the DPS. Furthermore, the image / video information may further include information from the NAL unit header.

[0162] On the other hand, to compensate for differences between the original image and the reconstructed image due to errors that occur during the compression encoding process, such as quantization, an in-loop filtering procedure can be performed on the reconstructed sample or reconstructed picture, as described above. As described above, in-loop filtering can be performed in the filter section of the encoding device and the filter section of the decoding device, and a deblocking filter, SAO, and / or adaptive loop filter (ALF) can be applied. For example, the ALF procedure can be performed after the deblocking filtering procedure and / or SAO procedure are completed. However, even in this case, the deblocking filtering procedure and / or SAO procedure may be omitted.

[0163] On the other hand, to improve coding efficiency, LMCS (luma mapping with chroma scaling) can be applied, as mentioned above. LMCS can also be called loop reshaping. To improve coding efficiency, the control of LMCS and / or the signaling of LMCS-related information can be performed hierarchically.

[0164] Figure 8 illustrates the hierarchical structure of a CVS according to one embodiment of this document. A CVS (coded video succession) may include an SPS (sequence parameter set), a PPS (picture parameter set), a tile group header, tile data, and / or a CTU(etc.). Here, the tile group header and tile data may also be called the slice header and slice data, respectively.

[0165] The SPS can primitively include flags to enable tools, as used in CVS. The SPS can also be referenced by a PPS containing information for parameters that vary per picture. Each encoded picture can contain one or more encoded rectangular domain tiles. These tiles can be grouped in a raster scan to form a tile group. Each tile group is encapsulated with header information called a tile group header. Each tile consists of a CTU containing encoded data, where the data can include original sample values, predicted sample values, and their luma and chroma components (luma predicted sample values ​​and chroma predicted sample values).

[0166] Figure 9 shows an exemplary LMCS structure according to one embodiment of this document. The LMCS structure 900 in Figure 9 may include an in-loop mapping portion 910 of the luma component based on an adaptive piecewise linear (adaptive PWL) model, and a luma-dependent chroma residual scaling portion 920 with respect to the chroma component. The inverse quantization and inverse transform 911, reconstruction 912, and intra-prediction 913 blocks of the in-loop mapping portion 910 represent the processes applied in the mapped (reshaped) domain. The loop filter 915, motion compensation or inter-prediction 917 blocks of the in-loop mapping portion 910, and the reconstruction 922, intra-prediction 923, motion compensation or inter-prediction 924, and loop filter 925 blocks of the chroma residual scaling portion 920 represent the processes applied in the original (non-mapped, unreshaped) domain.

[0167] As illustrated in Figure 9, once the LMCS is enabled, at least one of the following processes can be applied: inverse reshaping (mapping) process 914, forward reshaping (mapping) process 918, and chroma scaling process 921. For example, the inverse reshaping process can be applied to the (restored) luma sample (or luma sample or luma sample array) of the restored picture. The inverse reshaping process can be performed based on the piecewise function (inverse) index of the luma sample. The piecewise function (inverse) index can identify the piece (or part) to which the luma sample belongs. The output of the inverse reshaping process is the modified (restored) luma sample (or modified luma sample or modified luma sample array). The LMCS can be enabled or disabled at the tile group (or slice), picture, or higher level.

[0168] Forward reshaping and / or chroma scaling processes can be applied to generate a restored picture. A picture can contain both luma samples and chroma samples. A restored picture with luma samples can be called a restored luma picture, and a restored picture with chroma samples can be called a restored chroma picture. A combination of a restored luma picture and a restored chroma picture can be called a restored picture. Restored luma pictures can be generated based on a forward reshaping process. For example, if interpretation is applied to a current block, forward reshaping is applied to luma prediction samples derived based on the (restored) luma samples of the reference picture. Since the (restored) luma samples of the reference picture are generated based on an inverse reshaping process, forward reshaping can be applied to the luma prediction samples to derive reshaped (mapped) luma prediction samples. The forward reshaping process can be performed based on the piecewise function index of the luma prediction samples. The piecewise function index can be derived based on the values ​​of the Luma prediction samples or Luma samples of the reference picture used for interpretation. If intraprediction (or IBC (intra block copy)) is applied to the current block, forward mapping is not necessary because the inverse reshaping process has not yet been applied to the restored samples of the current picture. The (restored) Luma samples in the restored Luma picture are generated based on the reshaped Luma prediction samples and the corresponding Luma regi dual samples.

[0169] The restored chroma picture can be generated based on a chroma scaling process. For example, the (restored) chroma sample in the restored chroma picture is the chroma prediction sample and the chroma registration dual sample (c res It can be derived based on Chromares dual sample (cres ) is the (scaled) chroma residual sample (c resScale ) for the current block and is derived based on the chroma residual scaling factor (cScaleInv, which can be called varScale). The chroma residual scaling factor can be calculated based on the reshaped luma prediction sample values in the current block. For example, the scaling factor can be calculated based on the average luma value (ave(Y′ pred )) of the reshaped luma prediction sample values (Y′ pred ). For reference, in FIG. 9, the (scaled) chroma residual sample derived based on inverse transformation / inverse quantization is called c resScale , and the chroma residual sample derived by performing an (inverse) scaling procedure on the (scaled) chroma residual sample can be called c res .

[0170] FIG. 10 shows an LMCS structure according to another embodiment of this document. FIG. 10 will be described with reference to FIG. 9. Here, the differences between the LMCS structure of FIG. 10 and the LMCS structure 900 of FIG. 9 will be mainly described. The in-loop mapping part and the luma-dependent chroma residual scaling part of FIG. 10 can operate in the same / similar manner as the in-loop mapping part 910 and the luma-dependent chroma residual scaling part 920 of FIG. 9.

[0171] Referring to FIG. 10, the chroma residual scaling factor can be derived based on the luma restoration samples. In this case, the average luma value (avgY r ) can be obtained based on the adjacent luma restoration samples outside the restoration block that are not the internal luma restoration samples of the restoration block, and the average luma value (avgY rThe ChromaRegi dual scaling factor can be derived based on the following: where the adjacent chroma reconstruction sample is the adjacent chroma reconstruction sample of the current block, or the adjacent chroma reconstruction sample of the VPDU (virtual pipeline data unit) containing the current block. For example, if intra-prediction is applied to the target block, the reconstruction sample can be derived based on the prediction sample derived based on the intra-prediction. Alternatively, for example, if inter-prediction is applied to the target block, forward mapping can be applied to the prediction sample derived based on the inter-prediction, and the reconstruction sample can be generated based on the reshaped (or forward-mapped) chroma prediction sample.

[0172] The video / image information signaled via the bitstream may include LMCS parameters (information for the LMCS). LMCS parameters can be composed of HLS (high-level syntax, including slice header syntax), etc. A detailed description of LMCS parameters and their configuration will follow later. As mentioned above, the syntax tables described in this document (and the embodiments below) can be composed / encoded at the encoder end and signaled to the decoder end via the bitstream. The decoder can parse / decode the information for the LMCS (in the form of syntax components) in the syntax table. One or more embodiments described below can be combined. The encoder can encode the current picture based on the information for the LMCS, and the decoder can decode the current picture based on the information for the LMCS.

[0173] Luma component in-loop mapping can adjust the dynamic range of the input signal by redistributing codewords across the dynamic range to improve compression efficiency. For luma mapping, a forward mapping (reshaping) function (FwdMap) and an inverse mapping (reshaping) function (InvMap) corresponding to the forward mapping function (FwdMap) can be used. The forward mapping function (FwdMap) can be signaled using a sublinear model, for example, which may have 16 pieces or bins. These pieces may have the same length. In one example, the inverse mapping function (InvMap) is not signaled separately but can instead be derived from the forward mapping function (FwdMap). That is, the inverse mapping is a function of the forward mapping. For example, the inverse mapping function is a function that is symmetric to the forward mapping function with respect to y=x.

[0174] In-loop (luma) reshaping can be used to map input luma values ​​(samples) with modified values ​​in a reshaped domain. The reshaped values ​​are encoded and can then be remapped in the original (unmapped, unreshaped) domain after restoration. Chroma-Registry dual scaling can be applied to compensate for differences between luma and chroma signals. In-loop reshaping can be performed by specifying a high-level syntax for the reshaper model. The reshaper model syntax can signal a sublinear model (PWL model). A forward lookup table (FwdLUT) and / or inverse lookup table (InvLUT) can be derived based on the sublinear model. For example, if a forward lookup table (FwdLUT) is derived, an inverse lookup table (InvLUT) can be derived based on the forward lookup table (FwdLUT). The forward lookup table (FwdLUT) is the input luma value Y i The changed value Y r The mapping is performed, and the inverse lookup table (InvLUT) restores the value Y based on the changed value. r The restored value Y′ i It can be mapped to the recovered value Y′. i The input luma value Y i It can be derived based on this.

[0175] In one example, SPS may include the syntax shown in Table 2 below. The syntax in Table 2 may include sps_reshaper_enabled_flag as a tool enabling flag, where sps_reshaper_enabled_flag can be used to specify whether the reshaper is used in CVS (coded video sequence). In other words, sps_reshaper_enabled_flag is a flag that enables reshaping in SPS. In one example, the syntax in Table 2 is part of SPS.

[0176] [Table 2]

[0177] In one example, the semantics that sps_seq_parameter_set_id and sps_reshaper_enabled_flag can represent are shown in Table 3 below.

[0178] [Table 3]

[0179] In one example, a tile group header or slice header may include the syntax shown in Table 4 or Table 5 below.

[0180] [Table 4]

[0181] [Table 5]

[0182] The semantics of the syntax elements included in the syntax of Table 4 or Table 5 may include, for example, the items disclosed in the following table.

[0183] [Table 6]

[0184] [Table 7]

[0185] For example, when sps_reshaper_enabled_flag is parsed, the tile group header can parse additional data (e.g., information contained in Table 6 or Table 7) used to construct the lookup tables (FwdLUT and / or InvLUT). For this purpose, the state of the SPS reshaper flag can be checked in the slice header or tile group header. If sps_reshaper_enabled_flag is true (or 1), an additional flag, tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag), can be parsed. The purpose of tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) is to indicate the presence of a reshaper model. For example, if tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) is true (or 1), it can be indicated that a reshaper exists for the current tile group (or slice). If tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) is false (or 0), it can be indicated that a reshaper does not exist for the current tile group (or slice).

[0186] If a reshaper exists and is currently enabled in a tile group (or slice), the reshaper model (e.g., tile_group_reshaper_model() or slice_reshaper_model()) can be processed, along with an additional flag, tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag). tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) indicates whether the reshaper model is currently used in a tile group (or slice). For example, if tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is 0 (or false), it indicates that the reshaper model is not currently used in a tile group (or slice). If tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is 1 (or true), the reshaper model can be instructed to be currently used on the tile group (or slice).

[0187] For example, tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) is true (or 1) and tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is false (or 0). This means that a reshaper model exists but is not currently being used in the tile group (or slice). In such a case, the reshaper model can be used in the next tile group (or slice). Another example is tile_group_reshaper_enable_flag being true (or 1) and tile_group_reshaper_model_present_flag being false (or 0).

[0188] Once the reshaper model (e.g., tile_group_reshaper_model() or slice_reshaper_model()) and tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) are parsed, it can be determined (evaluated) whether the conditions necessary for chroma scaling exist. These conditions may include condition 1 (that the tile group / slice is not currently intra-encoded) and / or condition 2 (that the tile group / slice is not currently split into two separate coding quad-tree structures for luma and chroma, i.e., that the tile group / slice is not a dual-tree structure). If condition 1 and / or condition 2 are true, and / or tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is true (or 1), then tile_group_reshaper_chroma_residual_scale_flag (or slice_reshaper_chroma_residual_scale_flag) may be parsed. If tile_group_reshaper_chroma_residual_scale_flag (or slice_reshaper_chroma_residual_scale_flag) is enabled (1 or true), then it may be indicated that chroma residual scaling is now enabled for the tile group (or slice). If tile_group_reshaper_chroma_residual_scale_flag (or slice_reshaper_chroma_residual_scale_flag) is disabled (i.e., 0 or false), it can be indicated that chroma residual scaling should be disabled for the tile group (or slice).

[0189] The purpose of the aforementioned reshaping is to parse the data necessary to construct a lookup table (FwdLUT and / or InvLUT). In one example, the lookup table constructed based on the parsed data can divide the distribution of acceptable luma value ranges into multiple bins (e.g., 16). Thus, luma values ​​within a given bin can be mapped to modified luma values.

[0190] Figure 11 shows a graph illustrating an example of forward mapping. In Figure 8, only five bins are shown as an example.

[0191] Referring to Figure 11, the x-axis represents the input luma value, and the y-axis represents the modified output luma value. The x-axis is divided into five bins or pieces, each bin having a length L. That is, the five bins mapped to the modified luma value have the same length as each other. A forward lookup table (FwdLUT) can be constructed using data available in the tile group header (e.g., reshaper data), from which mapping is facilitated.

[0192] In one embodiment, output pivot points associated with the bin index can be calculated. The output pivot points can mark the minimum and maximum boundaries of the output range of the Luma codeword reshaping. The process of calculating the output pivot points can be performed based on a piecewise cumulative distribution function of the number of codewords. The output pivot range can be divided based on the maximum number of bins used and the size of the lookup table (FwdLUT or InvLUT). For example, the output pivot range can be divided based on the product of the maximum number of bins and the size of the lookup table. For example, if the product of the maximum number of bins and the size of the lookup table is 1024, the output pivot range can be divided into 1024 entries. The division of the output pivot range can be performed (applied or achieved) based on (utilized) a scaling factor. In one example, the scaling factor can be derived based on the following formula 1.

[0193]

number

[0194] In Equation 1, SF represents the scaling factor, and y1 and y2 represent the output pivot points corresponding to each bin. FP_PREC and c are predetermined constants. The scaling factor determined based on Equation 1 can be called the scaling factor for forward reshaping.

[0195] In other embodiments, in connection with inverse reshaping (inverse mapping), a defined range of bins (e.g., from reshaper_model_min_bin_idx to reshape_model_max_bin_idx) is patched with input reshaped pivot points and mapped inverse output pivot points (given by bin index * number of initial codewords) corresponding to mapped pivot points in a forward lookup table (FwdLUT). In other examples, the scaling factor (SF) can be derived based on the following equation 2.

[0196]

number

[0197] In Equation 2, SF represents the scaling factor, x1 and x2 represent the input pivot points, and y1 and y2 represent the output pivot points corresponding to each piece (bin). Here, the input pivot points are pivot points mapped based on a forward lookup table (FwdLUT), and the output pivot points are pivot points inversely mapped based on an inverse lookup table (InvLUT). Also, FP_PREC is a predetermined constant. FP_PREC in Equation 2 may be the same as or different from FP_PREC in Equation 1. The scaling factor determined based on Equation 2 can be called the scaling factor for inverse reshaping. During inverse reshaping, the input pivot points can be split based on the scaling factor in Equation 2. Based on the divided input pivot points, pivot values ​​corresponding to the minimum and maximum bin values ​​are specified for bin indices belonging to the range from 0 to the minimum bin index (reshaper_model_min_bin_idx) and / or from the minimum bin index (reshaper_model_min_bin_idx) to the maximum bin index (reshape_model_max_bin_idx).

[0198] Table 8 below shows the syntax of a reshaper model according to one embodiment. The reshaper model may be called an LMCS model. Here, the reshaper model is described exemplarily as a tile group reshaper, but this specification is not necessarily limited by this embodiment. For example, the reshaper model may be included in an APS, or the tile group reshaper model may also be called a slice reshaper model or LMCS data. The prefix reshaper_model or Rsp may also be used interchangeably with lmcs. For example, in the table and description below, reshaper_model_min_bin_idx, reshaper_model_delta_max_bin_idx, reshaper_model_max_bin_idx, RspCW, and RsepDeltaCW can be used interchangeably with lmcs_min_bin_idx, lmcs_delta_max_bin_idx, lmcs_max_bin_idx, lmcsCW, and lmcsDeltaCW, respectively.

[0199] [Table 8]

[0200] The semantics of the syntax elements included in the syntax of Table 8 above may include, for example, the items disclosed in the following table.

[0201] [Table 9-1]

[0202] [Table 9-2]

[0203] The inverse mapping procedure for Luma samples described in this document can be written in a standard document format as shown in the table below.

[0204] [Table 10]

[0205] The identification of the piecewise function index procedure for luma samples in this document can be described in the standard document format shown in the table below. In Table 11, idxYInv can be called the inverse mapping index, and the inverse mapping index can be derived based on the restored luma sample.

[0206] [Table 11]

[0207] Luma mapping can be performed based on the embodiments and examples described above, and the syntax and components contained herein are merely illustrative and not limited to the tables and formulas detailing the embodiments. Below, we describe how to perform chroma residual scaling (scaling of the chroma component of a residual sample) based on luma mapping.

[0208] Luma-dependent chroma-regi dual scaling is used to compensate for differences between luma samples and their corresponding chroma samples. For example, whether chroma-regi dual scaling is enabled can be signaled at the tile group level or slice group level. In one example, if luma mapping is enabled and dual tree partitioning is not currently applied to a tile group, an additional flag may be signaled to indicate whether luma-dependent chroma-regi dual scaling is enabled. In another example, if luma mapping is not used, or if dual tree partitioning is not currently applied to a tile group, luma-dependent chroma-regi dual scaling may be disabled. In yet another example, chroma-regi dual scaling may always be disabled for chroma blocks smaller than or equal to 4 in size.

[0209] ChromaRegi dual scaling can be performed based on the average value of the relevant luma prediction block (the luma component of the prediction block to which intra-prediction mode and / or inter-prediction mode are applied). Scaling calculations at the encoder and / or decoder ends can be implemented using fixed-point constant arithmetic based on the following equation 3.

[0210]

number

[0211] In the aforementioned equation 3, c' represents the scaled chroma-residual sample (scaled chroma component of the residual sample), c represents the chroma-residual sample (chroma component of the residual sample), s represents the chroma-residual scaling factor, and CSCALE_FP_PREC can represent a predetermined constant, for example, CSCALE_FP_PREC is 11.

[0212] Figure 12 is a flowchart illustrating a method for deriving a Chromaregi dual-scaling index according to one embodiment of this document. The method described in conjunction with Figure 12 can be performed based on the tables, formulas, variables, arrays, and functions contained in Figure 9 and its related description.

[0213] In step S1210, it can be determined whether the prediction mode is intra-prediction mode or inter-prediction mode based on the prediction mode information. If the prediction mode is intra-prediction mode, the current block or the prediction sample of the current block is considered to be in an already reshaped (mapped) region. If the prediction mode is inter-prediction mode, the current block or the prediction sample of the current block is considered to be in its original (unmapped, unreshaped) region.

[0214] In step S1220, if the prediction mode is intra-prediction mode, the mean of the current block (or the Luma prediction samples of the current block) can be calculated (derived). That is, the mean of the current block in the already reshaped region is directly calculated. The mean is also called the average value.

[0215] In step S1221, if the prediction mode is interprediction mode, forward reshaping (forward mapping) can be performed (applied) to the Luma prediction samples of the current block. Through forward reshaping, Luma prediction samples based on interprediction mode can be mapped from the original region to a reshaped region. In one example, forward reshaping of Luma prediction samples can be performed based on the reshaper model described with Table 4 above.

[0216] In step S1222, the mean of the forward-reshaped (forward-mapped) Luma prediction samples can be calculated (derived). That is, an averaging process can be performed on the forward-reshaped results.

[0217] In step S1230, the Chromaregi dual scaling index can be calculated. If the prediction mode is intra-prediction mode, the Chromaregi dual scaling index can be calculated based on the mean of the Luma prediction samples. If the prediction mode is inter-prediction mode, the Chromaregi dual scaling index can be calculated based on the mean of the forward-reshaped Luma prediction samples.

[0218] In one embodiment, the Chromaregi dual-scaling index can be calculated based on a for loop construct. The following table shows an exemplary for loop construct for deriving (calculating) the Chromaregi dual-scaling index.

[0219] [Table 12]

[0220] In Table 12 above, idxS represents the Chromaregi dual scaling index, idxS represents an index that identifies whether a Chromaregi dual scaling index satisfying the condition of the if statement has been obtained, S represents a predetermined constant, and MaxBinIdx represents the maximum acceptable bin index. ReshapPivot[idxS+1] can be derived based on Tables 8 and / or 9 above.

[0221] In one embodiment, the Chromaregi dual scaling factor can be derived based on the Chromaregi dual scaling index. Equation 4 below is an example of how to derive the Chromaregi dual scaling factor.

[0222]

number

[0223] In the above formula 4, s represents the ChromaRegi dual scaling factor, and ChromaScaleCoef is a variable (or array) derived based on Tables 8 and / or 9 described above.

[0224] As described above, the average luma value of the reference sample can be obtained, and a chroma-residual scaling factor can be derived based on the average luma value. As previously stated, scaling can be performed on the chroma-residual sample based on the chroma-residual scaling factor, and a chroma-restored sample can be generated based on the scaled chroma-residual sample.

[0225] In one embodiment of this paper, a signaling structure for efficiently applying the aforementioned LMCS is proposed. According to one embodiment of this paper, for example, LMCS data can be included in an HLS (e.g., APS), and an LMCS model (reshaper model) can be adaptively derived by signaling the referenced APS ID via header information (e.g., picture header, slice header), which is a lower level of the APS. The LMCS model can be derived based on LMCS parameters. Furthermore, for example, multiple APS IDs can be signaled via the header information, thereby allowing different LMCS models to be applied to each other on a block-by-block basis within the same picture / slice.

[0226] In one embodiment of this document, a method for efficiently performing the calculations required for LMCS is proposed. According to the semantics detailed in Table 9, a division operation by the piece length lmcsCW[i] is required to derive InvScaleCoeff[i]. However, if the piece length is not a power of 2, the division operation cannot be performed by bit shifting.

[0227] For example, the calculation of InvScaleCoeff requires up to 16 division operations per slice. According to Table 8 mentioned above, in the case of 10-bit coding, the range of lmcsCW[i] is from 8 to 511, so in order to implement division operations using lmcsCW[i] with a LUT, the size of the LUT must be 504. Also, in the case of 12-bit coding, the range of lmcsCW[i] is from 32 to 2047, so in order to implement division operations using lmcsCW[i] with a LUT, the size of the LUT must be 2016. In other words, division operations can be quite expensive to implement in hardware, and therefore, it is preferable to omit them if possible.

[0228] In one aspect of this embodiment, lmcsCW[i] can be restricted to a fixed number (or a pre-defined number or pre-determined number) multiple of a certain number. This allows the LUT (lookup table) for division operations to be reduced (in terms of its capacity or size). For example, if lmcsCW[i] is a multiple of 2, the size of the LUT used to substitute for division operations can be halved.

[0229] In another aspect of this embodiment, high internal bit depth coding is proposed. High internal bit depth coding is a higher-level condition for the range limitation of lmcsCW[i]. For example, if the coding bit depth is greater than 10, lmcsCW[i] can be limited to multiples of 1 << (BitDepthY - 10), where BitDepthY is the luma bit depth. This ensures that the possible values ​​of lmcsCW[i] do not change with the coding bit depth, and therefore the size of the LUT for the InvScaleCoeff calculation does not increase even with a high coding bit depth. In one example, for a 12-bit internal coding bit depth, the value of lmcsCW[i] can be limited to multiples of 4, so that the size of the LUT for substituting the division operation is the same as the size of the LUT used for 10-bit coding. This aspect can be implemented alone, but it can also be implemented in combination with the aspects described above.

[0230] In another aspect of this embodiment, lmcsCW[i] can be restricted to a narrower range. For example, lmcsCW[i] can be restricted to the range from (OrgCW>>1) to (OrgCW<<1)-1. In the case of 10-bit coding, the range of lmcsCW[i] is [32, 127], and InvScaleCoeff can be calculated with only a LUT having a size of 96.

[0231] In another aspect of this embodiment, lmcsCW[i] can be approximated to a number close to a power of 2 and used in the reshaper design. This allows the division operation in the inverse mapping procedure to be performed (replaced) by bit shifting.

[0232] In one embodiment of this document, a limitation on the LMCS codeword range is proposed. As shown in Table 9 above, the LMCS codeword values ​​are within the range of (OrgCW>>3) to (OrgCW<<3)-1. A wide range of LMCS piece lengths may result in a large difference between RspCW[i] and OrgCW, which could lead to visual degradation.

[0233] According to one embodiment of this document, it is proposed to restrict the codewords of the LMCS PWL mapping to a narrow range. For example, the range of lmcsCW[i] is from (OrgCW>>1) to (OrgCW<<1)-1.

[0234] In one embodiment of this paper, the use of a single chroma-regi dual scaling factor is proposed for chroma-regi dual scaling in LMCS. Existing methods for deriving the chroma-regi dual scaling factor derive the slope of each piece in the inverse chroma mapping as the scaling factor using the average value of the relevant chroma block. Furthermore, latency issues arose due to the procedure for identifying the piece-wise index that required the availability of the relevant chroma block. This is undesirable in hardware implementation. Through one embodiment of this paper, scaling in the chroma block becomes independent of the chroma block value and piece-wise index identification is no longer necessary. Therefore, the chroma-regi dual scaling procedure in LMCS can be performed without latency issues.

[0235] In one embodiment of this document, a single chroma scaling factor can be derived by both the encoder and decoder based on the LMCS information. When the LMCS LMCS model is received, the chroma regi dual scaling factor can be updated. For example, if the LMCS model is updated, the single chroma regi dual scaling factor can be updated.

[0236] The following table shows an example of how to obtain a single chroma scaling factor using this embodiment.

[0237] [Table 13]

[0238] Referring to Table 13, a single chroma scaling factor (e.g., ChromaScaleCoeff or ChromaScaleCoeffSingle) can be obtained by averaging the inverse chroma mapping slopes of all pieces within the range between lmcs_min_bin_idx and lmcs_max_bin_idx.

[0239] Figure 13 shows a linear fitting of pivot points according to one embodiment of this document. Pivot points P1, Ps, and P2 are shown in Figure 13. The following embodiments or examples thereof will be described with reference to Figure 13.

[0240] In one example of this embodiment, a single chroma scaling factor can be obtained based on a linear approximation of the chroma PWL mapping between pivot points lmcs_min_bin_idx and lmcs_max_bin_idx+1. That is, the inverse slope of the linear mapping can be used as the chroma dual scaling factor. For example, linear line 1 in Figure 13 is a straight line connecting pivot points P1 and P2. Referring to Figure 13, at P1, the input value is x1 and the mapped value is 0, and at P2, the input value is x2 and the mapped value is y2. The inverse slope (inverse scale) of linear line 1 is (x2-x1) / y2, and the single chroma scaling factor ChromaScaleCoeffSingle can be calculated based on the input and mapped values ​​of pivot points P1 and P2, and the following formula.

[0241]

number

[0242] In equation 5, CSCALE_FP_PREC represents the shift factor, which is a predetermined constant. For example, CSCALE_FP_PREC is 11.

[0243] In another example according to this embodiment, referring to Figure 13, at pivot point Ps, the input value is min_bin_idx+1 and the mapped value is ys. Thus, the inverse slope (inverse scale) of linear line 1 can be calculated as (xs-x1) / ys, and the single chroma scaling factor ChromaScaleCoeffSingle can be calculated based on the input and mapped values ​​of pivot point P1 and Ps, and the following formula.

[0244]

number

[0245] In Equation 6, CSCALE_FP_PREC represents the shift factor (the factor for bit shifting), which is a predetermined constant, for example. In one example, CSCALE_FP_PREC is 11, and bit shifting to the inverse scale can be performed based on CSCALE_FP_PREC.

[0246] In another example according to this embodiment, the single chroma residual scaling factor can be derived based on a linear approximation line. An example for deriving the linear approximation line can include a linear connection of pivot points (e.g., lmcs_min_bin_idx, lmcs_max_bin_idx + 1). For example, the linear trend result can be represented by the codewords of PWL mapping. The mapped value y2 at P2 is the sum of the codewords of all bins (pieces), and the difference (x2 - x1) between the input value at P2 and the input value at P1 is OrgCW * (lmcs_max_bin_idx - lmcs_min_bin_idx + 1) (OrgCW refers to Table 9 described above). The following table shows an example of obtaining the single chroma scaling factor according to the above-described embodiment.

[0247]

Table 14

[0248] Referring to Table 14, the single chroma scaling factor (e.g., ChromaScaleCoeffSingle) can be obtained from two pivot points (i.e., lmcs_min_bin_idx, lmcs_max_bin_idx). For example, the inverse slope of the linear mapping can be used as the chroma scaling factor.

[0249] In another example of this embodiment, the single chroma scaling factor can be obtained by linear fitting of pivot points to minimize the error (or mean squared error) between the linear fitting and the existing PWL mapping. This example is more accurate than simply connecting the two pivot points of lmcs_min_bin_idx and lmcs_max_bin_idx. There can be various methods to find the optimal linear mapping, and an example will be described below.

[0250] In one example, the parameters b1 and b0 of the linear fitting equation y = b1 * x + b0 for minimizing the sum of the least squares errors can be calculated based on the following Equation 7 and / or Table 8.

[0251]

Number

[0252]

Number

[0253] Here, x represents the original luma value, and y represents the reshaped luma value. Specifically JPEG0007886476000024.jpg1391 each represent the average of x and y, and xi and yi represent the values of the i-th pivot point.

[0254] Referring to FIG. 13, another approximation for identifying the linear mapping can be given as follows:

[0255] - By connecting the pivot points of the PWL mapping at -lmcs_min_bin_idx and lmcs_max_bin_idx + 1, obtain Linear Line 1 and calculate lmcs_pivots_linear[i] on the linear line having an input value that is a multiple of OrgCW

[0256] - Using Linear Line 1 and the PWL mapping, sum the differences between the mapped values of the pivot points

[0257] - Obtain the average difference (avgDiff)

[0258] - Adjust the last pivot point of the linear line by the average difference (e.g., 2 * avgDiff)

[0259] - Use the inverse slope of the adjusted linear line as the Chromaregi dual scale.

[0260] Through the linear fitting described above, the chromascaling factor (i.e., the inverse slope of forward mapping) can be derived (obtained) based on Equation 9 or Equation 10 below.

[0261]

number

[0262]

number

[0263] In the formula described above, lmcs_pivots_lienar[i] is the mapped value of the linear mapping. Through the linear mapping, all pieces of the PWL mapping between the minimum and maximum bin indices can have the same LMCS codeword (lmcsCW). That is, lmcs_pivots_linear[lmcs_min_bin_idx+1] is the same as lmcsCW[lmcs_min_bin_idx].

[0264] Furthermore, in equations 9 and 10, CSCALE_FP_PREC represents the shift factor (the factor for bit shifting), and for example, CSCALE_FP_PREC is a predetermined constant. In one example, CSCALE_FP_PREC is 11.

[0265] It is not necessary to calculate the average of the corresponding luma block via the single chroma residual scaling factor (ChromaScaleCoeffSingle), and it is not necessary to reduce the index with PWL linear mapping. Therefore, the coding efficiency using chroma residual scaling can be increased.

[0266] In other embodiments of this document, the encoder can determine parameters regarding the single chroma scaling factor and can signal said parameters to the decoder. Through signaling, the encoder can make other information available in the encoder available for deriving the chroma residual scaling factor. This embodiment aims to remove the chroma residual scaling latency problem.

[0267] For example, the procedure for identifying the linear mapping used to determine the chroma residual scaling factor can be given as follows:

[0268] - By concatenating the pivot points of the PWL mapping at -lmcs_min_bin_idx and lmcs_max_bin_idx + 1, obtain linear line 1 and calculate lmcs_pivots_linear[i] on the linear line with input values that are multiples of OrgCW

[0269] - Use linear line 1 and the pivot points of the luma PWL mapping to obtain the weighted sum of the differences between the mapped values of the pivot points

[0270] - Obtain the weighted average difference (avgDiff)

[0271] - Adjust the last pivot point of linear line 1 by the weighted average difference (e.g., 2 * avgDiff)

[0272] - Use the inverse slope of the adjusted linear line as the Chromaregi dual scale.

[0273] The table below shows exemplary syntax for signaling y values ​​to derive the chroma scaling factor.

[0274] [Table 15]

[0275] In Table 15, the syntax element lmcs_chroma_scale can specify the single chroma (residual) scaling factor used for LMCS chroma regi dual scaling (ChromaScaleCoeffSingle=lmcs_chroma_scale). That is, information about the chroma regi dual scaling factor can be directly signaled, and the signaled information can be derived as the chroma regi dual scaling factor. Alternatively, the value of the signaled information about the chroma regi dual scaling factor can be (directly) derived as the value of the single chroma regi dual scaling factor. Here, the syntax element lmcs_chroma_scale can be signaled together with other LMCS data (e.g., absolute values ​​of codewords, syntax elements related to signs, etc.).

[0276] Alternatively, the encoder can signal only the parameters necessary to derive the ChromaRegi dual scaling factor to the decoder. To derive the ChromaRegi dual scaling factor in the decoder, an input value x and a mapped value y are required. The x value represents the bin length and is already known to the decoder end, so it does not need to be signaled. Therefore, only the y value needs to be signaled for the ChromaRegi dual scaling factor derivation. Here, the y value is the mapped value of any pivot point in the linear mapping (e.g., the mapped value of P2 or Ps in Figure 13).

[0277] The table below shows an example of signaling the mapped values ​​for chroma regi dual scaling factor derivation.

[0278] [Table 16]

[0279] [Table 17]

[0280] One of the syntaxes in Tables 16 and 17 mentioned above can be used to signal the y value at any linear pivot point specified by the encoder and decoder. That is, the encoder and decoder can derive the y value using the same syntax as each other.

[0281] First, the embodiments shown in Table 16 will be described. In Table 16, lmcs_cw_linear can represent a value mapped to Ps or P2. That is, in the embodiments shown in Table 16, a fixed number can be signaled via lmcs_cw_linear.

[0282] In one example according to this embodiment, if lmcs_cw_linear represents a value mapped to one bin (i.e., lmcs_pivots_linear[lmcs_min_bin_idx+1] in Ps in Figure 13), the chroma scaling factor can be derived based on the following formula.

[0283]

number

[0284] In another example according to this embodiment, if lmcs_cw_linear is lmcs_max_bin_idx+1 (i.e., lmcs_pivots_linear[lmcs_max_bin_idx+1] at P2 in Figure 13), the chroma scaling factor can be derived based on the following formula.

[0285]

number

[0286] In the formula described above, CSCALE_FP_PREC represents the shift factor (the factor for bit shifting), and for example, CSCALE_FP_PREC is a predetermined constant. In one example, CSCALE_FP_PREC is 11.

[0287] Next, embodiments shown in Table 17 will be described. In this embodiment, lmcs_cw_linear can also be signaled as delta values ​​associated with a fixed number (i.e., lmcs_delta_abs_cw_linear, lmcs_delta_sign_cw_linear_flag). In one example of this embodiment, if lmcs_cw_linear represents the mapped value in lmcs_pivots_linear[lmcs_min_bin_idx+1] (i.e., Ps in Figure 13), then lmcs_cw_linear_delta and lmcs_cw_linear can be derived based on the following formulas.

[0288]

number

[0289]

number

[0290] In another example of this embodiment, if lmcs_cw_linear represents the mapped value in lmcs_pivots_linear[lmcs_max_bin_idx+1] (i.e., P2 in Figure 13), then lmcs_cw_linear_delta and lmcs_cw_linear can be derived based on the following formulas.

[0291]

number

[0292]

number

[0293] In the formula mentioned above, OrgCW is a value derived based on Table 9 mentioned earlier.

[0294] Figure 14 shows an example of linear reshaping (or linear reshaper, linear mapping) according to one embodiment of this document. Specifically, one embodiment of this document proposes the use of a linear reshaper in LMCS. For example, the example in Figure 14 relates to forward linear reshaping (mapping).

[0295] In existing examples, LMCS can use piecewise linear mapping with 16 fixed pieces. This can increase the complexity of the reshaper design because abrupt transitions between pivot points inevitably lead to degradation. Furthermore, in the case of inverse luma mapping of the reshaper, the piecewise function index must be identified. The piecewise function index identification procedure is an iterative process that involves many comparison execution steps. Additionally, a luma piecewise index identification procedure is required for chroma regi dual scaling using the corresponding luma block mean. This not only introduces a complexity problem but can also cause latency in chroma regi dual scaling that depends on the reconstruction of the entire luma block. To solve these problems, the use of a linear reshaper in LMCS has been proposed, and a detailed explanation of the linear reshaper is provided below.

[0296] Referring to Figure 14, a linear reshaper can have two pivot points (i.e., P1, P2). P1 and P2 can represent inputs and mapped values, for example, P1 is (minInput, 0) and P2 is (maxInput, maxMapped), where minInput represents the minimum input value and maxInput represents the maximum input value. If the input value is less than or equal to minInput, it is mapped to 0, and if the input value is greater than maxInput, it is mapped to maxMapped. Input (lumen) values ​​between minInput and maxInput can be linearly mapped to other values. Figure 14 shows an example of mapping. The pivot points P1 and P2 can be determined by an encoder, for which linear fitting can be used to approximate a piecewise linear mapping.

[0297] There are various ways to signal a linear reshaper. In one example of signaling a linear reshaper, each luma range can be divided into an equal number of bins. That is, luma mappings between the minimum and maximum bins can be evenly distributed (achieved). For example, all bins can have the same LMCS codeword (lmcsCW). For this purpose, the minimum and maximum bin indices can be signaled. Additionally, only one set of reshape_model_bin_delta_abs_CW (or reshaper_model_delta_abs_CW, lmcs_delta_abs_CW) and reshaper_model_bin_delta_sign_CW_flag (or reshaper_model_delta_sign_CW_flag, lmcs_delta_sign_CW_flag) needs to be signaled.

[0298] The following table exemplifies the syntax and semantics used to signal a linear reshaper in this example.

[0299] [Table 18]

[0300] [Table 19]

[0301] Referring to Tables 18 and 19, the syntax element log2_lmcs_num_bins_minus4 is information about the number of bins. Based on this information, the number of bins can be signaled, thus allowing for better control of the minimum and maximum pivot points. In other existing examples, encoders and / or decoders can derive (explicitly) a (fixed) number of bins without signaling, for example, the number of bins is derived to be 16 or 32. However, as illustrated in Tables 18 and 19, log2_lmcs_num_bins_minus4 plus 4 can represent the logarithm of the number of bins. The number of bins derived based on the syntax element ranges from 4 to the value of the luma bit depth (BitDepthY).

[0302] In Table 19 above, ScaleCoeffSingle can be called a single luma forward scaling factor, and InvScaleCoeffSingle can be called a single luma inverse scaling factor. A (forward) mapping can be performed on predicted luma samples based on the single luma forward scaling factor, and a (inverse) mapping can be performed on reconstructed luma samples based on the single luma inverse scaling factor. ChromaSclaeCoeffSingle can be called a single chroma regi dual scaling factor, as previously mentioned. ScaleCoeffSingle, InvScaleCoeffSingle, and ChromaSclaeCoeffSingle can be used for forward luma mapping, inverse luma mapping, and chroma regi dual scaling, respectively. ScaleCoeffSingle, InvScaleCoeffSingle, and ChromaSclaeCoeffSingle can be applied uniformly as a single factor to all bins (16 bins PWL mappings).

[0303] Referring to Table 19 above, FP_PREC and CSCALE_FP_PREC are constants for bit shifting. FP_PREC and CSCALE_FP_PREC may or may not be the same. For example, FP_PREC may be greater than or equal to CSCALE_FP_PREC. In one example, both FP_PREC and CSCALE_FP_PREC are 11. In another example, FP_PREC is 15 and CSCALE_FP_PREC is 11.

[0304] In another example of signaling linear reshapers, the LMCS codeword (lmcsCWlinearALL) can be derived based on the following formula. In this example, the linear reshaping syntax elements signaled by the syntax in Table 18 above can also be used. The following table shows an example of the semantics described by this example.

[0305] [Table 20]

[0306] Referring to Table 20 above, FP_PREC and CSCALE_FP_PREC are constants for bit shifting. FP_PREC and CSCALE_FP_PREC may or may not be the same. For example, FP_PREC may be greater than or equal to CSCALE_FP_PREC. In one example, both FP_PREC and CSCALE_FP_PREC are 11. In another example, FP_PREC is 15 and CSCALE_FP_PREC is 11.

[0307] In Table 20, lmcs_max_bin_idx can be used interchangeably with LmcsMaxBinIdx. lmcs_max_bin_idx and LmcsMaxBinIdx can be derived by referring to Table 19, based on the lmcs_delta_max_bin_idx syntax in Table 18. Thus, Table 20 can be analyzed by referring to Table 19.

[0308] In other embodiments of this document, other examples of methods for signaling a linear reshaper may be proposed. The pivot points P1 and P2 of the linear reshaper model can be explicitly signaled. The following table shows an example of syntax and semantics for explicitly signaling a linear reshaper model by this example.

[0309] [Table 21]

[0310] [Table 22]

[0311] Referring to Tables 21 and 22, the input value of the first pivot point can be derived based on the syntax element lmcs_min_input, and the input value of the second pivot point can be derived based on the syntax element lmcs_max_input. The mapped value of the first pivot point is a predetermined value (a value known to both the encoder and decoder), for example, 0. The mapped value of the second pivot point can be derived based on the syntax element lmcs_max_mapped. That is, the linear reshaper model can be explicitly (directly) signaled based on the information signaled based on the syntax in Table 21.

[0312] Alternatively, lmcs_max_input and lmcs_max_mapped can be signaled as delta values. The table below shows an example of syntax and semantics for signaling a linear reshaper model as a delta value.

[0313] [Table 23]

[0314] [Table 24]

[0315] Referring to Table 24 above, the input value of the first pivot point can be derived based on the syntax element lmcs_min_input. For example, lmcs_min_input can have a mapped value of 0. lmcs_max_input_delta can indicate the difference between the input value of the second pivot point and the maximum luma value (i.e., (1 << bitdepthY) - 1). lmcs_max_mapped_delta can indicate the difference between the mapped value of the second pivot point and the maximum luma value (i.e., (1 << bitdepthY) - 1).

[0316] According to one embodiment of this document, forward mapping for luma prediction samples, inverse mapping for luma restoration samples, and chroma residual scaling can be performed based on the examples described above for the linear reshaper. In one example, only one inverse scaling factor is required for inverse scaling for luma (restoration) samples (pixels) in the linear reshaper-based inverse mapping. This is also the case for forward mapping and chroma residual scaling. That is, the step of determining ScaleCoeff[i], InvScaleCoeff[i] and ChromaScaleCoeff[i] for bin index i can be replaced by using only one factor. Here, one factor can mean that the (forward) slope or inverse slope of the linear mapping is represented in fixed-point. In one example, the inverse luma mapping scaling factor (the inverse scaling factor in the inverse mapping for luma restoration samples) can be derived based on at least one of the following equations.

[0317]

Equation

[0318]

Equation

[0319]

number

[0320] The lmcsCWLinear in formula 17 can be derived from Tables 18 and 19 mentioned above. The lmcsCWLinearALL in formula 18 and Table 19 can be derived from at least one of Tables 20 to 24 mentioned above. In formula 17 or formula 18, OrgCW can be derived from Table 9 or Table 19.

[0321] The following table illustrates the formulas and syntax (conditional statements) for the forward mapping procedure to Luma samples (i.e., Luma prediction samples) in picture restoration. In the following table and formulas, FP_PREC is a constant for bit shifting and is a predetermined value. For example, FP_PREC is 11 or 15.

[0322] [Table 25]

[0323] [Table 26]

[0324] Table 25 is for deriving the luma samples forward-mapped by the luma mapping procedure based on Table 8 and Table 9 described above. That is, Table 25 can be explained together with Table 8 and Table 9. In Table 25, the forward-mapped luma (prediction) samples PredmAPSamples[i][j] as output can be derived from the luma (prediction) samples predSamples[i][j] as input. idxY in Table 25 can be called the (forward) mapping index, and the mapping index can be derived based on the predicted luma samples.

[0325] Table 26 is for deriving the luma samples forward-mapped from the luma mapping by applying the linear reshaper. For example, lmcs_min_input, lmcs_max_input, lmcs_max_mapped, and ScaleCoeffSingle in Table 26 can be derived by at least one of Tables 21 to 24. In Table 26, when lmcs_min_input < predSamples[i][j] < lmcs_max_input, the forward-mapped luma (prediction) samples PredmAPSamples[i][j] as output can be derived from the luma (prediction) samples predSamples[i][j] as input. Through the comparison between Table 25 and Table 26, the change from the existing LMCS by applying the linear reshaper can be seen from the perspective of forward mapping.

[0326] The following equations explain the inverse mapping procedure for luma samples (i.e., luma restoration samples). In the following equations, the lumaSample as input is the (pre-modification) luma restoration sample before inverse mapping. The invSample as output is the inversely mapped (modified) luma restoration sample. In other cases, the clipped invSample is also called the modified luma restoration sample.

[0327]

Equation

[0328]

number

[0329] Referring to Equation 20, the index idxInv can be derived based on Table 11 mentioned above. That is, Equation 20 is for deriving the inverse-mapped luma sample in the luma mapping procedure based on Tables 8 and 9 mentioned above. Equation 20 can be explained together with Table 10 mentioned above.

[0330] Equation 21 is used to derive the inversely mapped luma sample in luma mapping by applying a linear reshaper. For example, lmcs_min_input in Equation 21 can be derived from at least one of Tables 21 to 24. By comparing Equation 20 and Equation 21, the changes from the existing LMCS due to the application of the linear reshaper can be seen from a forward mapping perspective.

[0331] Based on the example of the linear reshaper described above, the piece-wise index identification procedure can be omitted. That is, in this example, since there is only one piece with a reshaped lumapixel, the piece-wise index identification procedure used for inverse luma mapping and ChromaRegi dual scaling can be eliminated. This reduces the complexity of inverse luma mapping. In addition, the latency problem caused by relying on luma piece-wise index identification can be eliminated during ChromaRegi dual scaling.

[0332] The embodiments of the linear reshaper described above can provide the following advantages for LMCS: i) Simplify the encoder reshaper design to prevent degradation due to sudden changes occurring between piecewise linear pieces. ii) Simplify the decoder inverse mapping procedure by eliminating the piecewise index identification procedure. iii) Eliminate latency issues in ChromaRegi dual scaling caused by dependence on the relevant Luma block by eliminating the piecewise index identification procedure. iv) Reduce signaling overhead and make frequent reshaper updates more feasible. v) Eliminate loops from many parts that require 16-piece loops (e.g., for constructs). For example, the number of division operations by lmcsCW[i] to derive InvScaleCoeff[i] can be reduced to 1.

[0333] In other embodiments of this document, an LMCS based on flexible bins is proposed. Here, "flexible bins" can mean that the number of bins is not fixed to a predetermined number. In one existing embodiment, the number of bins in the LMCS was fixed at 16, and the 16 bins were evenly distributed with respect to the input sample values. In this embodiment, a flexible number of bins is proposed, and the pieces (bins) are not evenly distributed with respect to the original pixel values.

[0334] The following table exemplifies the syntax of LMCS data (data fields) according to this embodiment and the semantics of the syntax elements contained therein.

[0335] [Table 27]

[0336] [Table 28]

[0337] Referring to Table 27, information lmcs_num_bins_minus1 regarding the number of bins can be signaled. Referring to Table 28, lmcs_num_bins_minus1 + 1 is the same as the number of bins, and thus the number of bins is within the range from 1 to (1 << BitDepthY)-1. For example, lmcs_num_bins_minus1 or lmcs_num_bins_minus1 + 1 is a multiple of 2.

[0338] In the embodiments described with Tables 27 and 28, the number of pivot points can be derived based on lmcs_num_bins_minus1 (information regarding the number of bins) regardless of whether the reshaper is linear (signaling of lmcs_num_bins_minus1), and the input values and mapped values of the pivot points (LmcsPivot_input[i], LmcsPivot_mapped[i]) can be derived based on the summation of the signaled codeword values (lmcs_delta_input_cw[i], lmcs_delta_mapped_cw[i]) (where the initial input value LmcsPivot_input[0] and the initial output value LmcsPivot_mapped[0] are 0).

[0339] FIG. 15 shows an example of linear forward mapping in an embodiment of this document. FIG. 16 shows an example of inverse forward mapping in an embodiment of this document.

[0340] The embodiments shown in Figures 15 and 16 propose a method to support both regular LMCS and linear LMCS. In one example according to this embodiment, regular LMCS and / or linear LMCS can be indicated based on the syntax element lmcs_is_linear. After the linear LMCS line is determined in the encoder, the mapped value (e.g., the mapped value in pL in Figures 15 and 16) is divided into equal pieces (e.g., LmcsMaxBinIdx - lmcs_min_bin_idx + 1). The codeword in binLmcsMaxBinIdx can be signaled using the syntax relating to the aforementioned lmcs data or reshaper model.

[0341] The following table exemplifies the syntax of LMCS data (data fields) and the semantics of the syntax elements contained therein according to an example of this embodiment.

[0342] [Table 29]

[0343] [Table 30-1]

[0344] [Table 30-2]

[0345] The following table exemplifies the syntax of LMCS data (data fields) and the semantics of the syntax elements contained therein according to other examples of this embodiment.

[0346] [Table 31]

[0347] [Table 32-1]

[0348] [Table 32-2]

[0349] Referring to Tables 29 to 32 above, if lmcs_is_linear_flag is true, all lmcsDeltaCW[i] between lmcs_min_bin_idx and LmcsMaxBinIdx can have the same value. That is, all pieces between lmcs_min_bin_idx and LmcsMaxBinIdx can have the same value for lmcsCW[i]. The scale, inverse scale, and chroma scale of all pieces between lmcs_min_bin_idx and lmcsMaxBinIdx are the same. If the linear reshaper is true, it is not necessary to derive the piece index, and the scale and inverse scale can be used on one of the pieces.

[0350] The following table illustrates the procedure for identifying piecewise indices according to this embodiment.

[0351] [Table 33]

[0352] According to other embodiments of this document, the application of regular 16-piece PWL LMCS and linear LMCS can depend on high-level syntax (e.g., sequence level).

[0353] The following table exemplifies the syntax of SPS according to this embodiment and the semantics for the syntactic elements contained therein.

[0354] [Table 34]

[0355] [Table 35]

[0356] Referring to Tables 34 and 35, the availability of regular LMCS and / or linear LMCS can be determined (signaled) by the syntax elements included in SPS. Referring to Table 35, based on the syntax element sps_linear_lmcs_enabled_flag, one of either regular LMCS or linear LMCS is available on a sequence basis.

[0357] In addition, whether one or two of linear LMCS or regular LMCS are available depends on the pull file level. For example, for a particular pull file (e.g., an SDR pull file), only linear LMCS may be allowed, for another pull file (e.g., an HDR pull file), only regular LMCS may be allowed, and for yet another pull file, both regular LMCS and / or linear LMCS may be allowed.

[0358] According to other embodiments of this document, the LMCS piecewise index identification procedure can be used with inverse luma mapping and Chromaregi dual scaling. In this embodiment, the piecewise index identification procedure can be used for blocks on which Chromaregi dual scaling is available, and the identification procedure can also be used for all luma samples within a reshaped (mapped) region. The purpose of this embodiment is to reduce the complexity for deriving the index.

[0359] The table below shows the identification procedure (derivation procedure) for existing piecewise function indices.

[0360] [Table 36]

[0361] In one example, the piecewise index identification procedure can classify an input sample into at least two categories. For example, the input sample can be classified into three categories: a first, a second, and a third category. For example, the first category may represent samples (values) smaller than LmcsPivot[lmcs_min_bin_idx+1], the second category may represent samples (values) larger than or equal to LmcsPivot[LmcsMaxBinIdx], and the third category may represent samples (values) between LmcsPivot[lmcs_min_bin_idx+1] and LmcsPivot[LmcsMaxBinIdx].

[0362] In this embodiment, a method is proposed to optimize the identification procedure by eliminating category classification. Since the input to the piecewise index identification procedure is a reshaped (mapped) Luma value, there should be no values ​​that exceed the mapped values ​​at the pivot points lmcs_min_bin_idx and LmcsMaxBinIdx+1. Therefore, the conditional procedure of classifying samples by category can be omitted in the existing piecewise index identification procedure, and a specific example is described below with a table.

[0363] In one example according to this embodiment, the identification procedure included in Table 36 can be replaced by one of the following Tables 37 or 38. Referring to Tables 37 and 38, the first two categories in Table 36 can be deleted, and the boundary value (second boundary value or ending point) in the iterative loop (for construct) for the last category can be modified from LmcsMaxBinIdx to LmcsMaxBinIdx+1. That is, the identification procedure can be simplified, and the complexity for deriving the piecewise index can be reduced. Therefore, LMCS-related coding can be performed efficiently with this embodiment.

[0364] [Table 37]

[0365] [Table 38]

[0366] Referring to Table 37, the comparison procedure corresponding to the condition in the if clause (the formula corresponding to the condition in the if clause) can be repeatedly executed for all bin indices from the minimum bin index to the maximum bin index. When the formula corresponding to the condition in the if clause is true, the bin index can be derived as an inverse mapping index for inverse luma mapping (or an inverse scaling index for chroma regi dual scaling). Based on the inverse mapping index, a corrected restored luma sample (or a scaled chroma regi dual sample) can be derived.

[0367] The following drawings have been prepared to illustrate a specific example of this specification. The names of specific devices and signals / messages / fields shown in the drawings are presented illustratively, and the technical features of this specification are not limited to the specific names used in the following drawings.

[0368] Figures 17 and 18 schematically show an example of a video / image encoding method and related components according to embodiments (etc.) of this document. The method disclosed in Figure 17 can be performed by the encoding apparatus disclosed in Figure 2. Specifically, for example, S1700 and S1710 in Figure 17 can be performed by the prediction unit 220 of the encoding apparatus, S1720 can be performed by the residual processing unit 230 of the encoding apparatus, S1730 can be performed by the prediction unit 220 or the residual processing unit 230 of the encoding apparatus, S1740 can be performed by the residual processing unit 230 or the addition unit 250 of the encoding apparatus, S1750 can be performed by the residual processing unit 230 of the encoding apparatus, and S1760 can be performed by the entropy encoding unit 240 of the encoding apparatus. The method disclosed in Figure 17 may include embodiments described above in this document.

[0369] As shown in Figure 17, the encoding device can derive an interprediction mode for the current block in the current picture (S1700). The encoding device can derive at least one of the various interprediction modes disclosed in this document.

[0370] The encoding device can generate predicted luma samples based on the interprediction mode (S1710). The encoding device can generate predicted luma samples by making predictions for the original samples currently included in the block.

[0371] The encoding device can derive a predicted chroma sample. The encoding device can derive the residual chroma sample based on the original chroma sample and the predicted chroma sample of the current block. For example, the encoding device can derive the residual chroma sample based on the difference between the predicted chroma sample and the original chroma sample.

[0372] The encoding device can derive bins and LMCS codewords for lumens mapping. The encoding device can derive the bins and / or LMCS codewords based on SDR or HDR.

[0373] The encoding device can derive LMCS-related information (S1720). The LMCS-related information may include bins and LMCS codewords for lm mapping.

[0374] The encoding device can generate predicted luma samples mapped based on a mapping procedure for luma samples (S1730). The encoding device can generate predicted luma samples mapped based on bins and / or LMCS codewords for luma mapping. For example, the encoding device can derive input values ​​and mapping values ​​(output values) for pivot points for luma mapping, and generate predicted luma samples mapped based on the input values ​​and mapping values. As an example, the encoding device can derive a mapping index (idxY) based on a first predicted luma sample, and generate a first mapped predicted luma sample based on the input values ​​and mapping values ​​of the pivot point corresponding to the mapping index. As another example, linear mapping (linear reshaping, linear LMCS) may be used, and predicted luma samples mapped based on a forward mapping scaling factor derived from two pivot points in linear mapping may be generated, and thus the index derivation procedure may be omitted by linear mapping.

[0375] The encoding device can generate scaled resistive chroma samples. Specifically, the encoding device can derive a chroma-residual scaling factor and generate scaled resistive chroma samples based on the chroma-residual scaling factor. Here, chroma-residual scaling at the encoding end is also called forward chroma-residual scaling. Therefore, the chroma-residual scaling factor derived by the encoding device can be called the forward chroma-residual scaling factor, and a forward-scaled resistive chroma sample can be generated.

[0376] The encoding device can generate a reconstructed luma sample. The encoding device can generate a reconstructed luma sample based on the mapped predicted luma sample. Specifically, the encoding device can sum the aforementioned residual luma sample with the mapped predicted luma sample and generate a reconstructed luma sample based on the summation result.

[0377] The encoding device can generate a corrected restored luma sample based on an inverse mapping procedure for the luma sample (S1740). The encoding device can generate a corrected restored luma sample based on the bins, LMCS codewords, and the restored luma sample for the luma mapping. The encoding device can generate the corrected restored luma sample via an inverse mapping procedure for the restored luma sample. For example, the encoding device can derive an inverse mapping index (e.g., invYIdx) based on a mapping value (e.g., LmcsPivot[i], i=lmcs_min_bin_idx...LmcsMaxBinIdx+1) assigned to each restored luma sample and / or bin index in the inverse mapping procedure. The encoding device can generate a corrected restored luma sample based on a mapping value (LmcsPivot[invYIdx]) assigned to the inverse mapping index.

[0378] The encoding device can generate a residual luma sample based on the mapped predicted luma sample. For example, the encoding device can derive a residual luma sample based on the difference between the mapped predicted luma sample and the original luma sample.

[0379] The encoding device can derive residual information (S1750). For example, the encoding device can generate residual information based on the mapped predicted chroma sample and the corrected restored chroma sample. For example, the encoding device can derive residual information based on the scaled residual chroma sample and / or the residual chroma sample. The encoding device can derive conversion coefficients based on the scaled residual chroma sample and / or the residual chroma sample. For example, the conversion procedure may include at least one of DCT, DST, GBT, or CNT. The encoding device can derive quantized conversion coefficients based on a quantization procedure for the conversion coefficients. The quantized conversion coefficients may have a one-dimensional vector form based on the coefficient scan order. The encoding device can generate residual information representing the quantized conversion coefficients. The residual information can be generated via various encoding methods such as exponential Golomb, CAVLC, CABAC, etc.

[0380] The encoding device can encode image / video information (S1760). The image information may include information about LMCS data and / or residual information. For example, the LMCS-related information may include information about a linear LMCS. In one example, at least one LMCS codeword can be derived based on the information about the linear LMCS. The encoded video / image information can be output in bitstream form. The bitstream can be transmitted to a decoding device via a network or storage medium.

[0381] The aforementioned image / video information may include a variety of information according to the embodiments described herein. For example, the aforementioned image / video information may include information disclosed in at least one of the aforementioned Tables 1 to 38.

[0382] In one embodiment, a minimum bin index and a maximum bin index can be derived based on information about the bins. For example, a mapping value based on bin indices from the minimum bin index to the maximum bin index can be derived based on information about the LMCS codeword. The mapping value may include a first mapping value (e.g., LmcsPivot[idxY]) and a second mapping value (e.g., LmcsPivot[idxYInv]). In the mapping procedure for the luma sample, a mapping index can be derived based on the predicted luma sample, and the mapped predicted luma sample can be generated using the first mapping value based on the mapping index. In the inverse mapping procedure for the luma sample, an inverse mapping index can be derived based on the mapping value based on bin indices from the minimum bin index to the maximum bin index, and the modified restored luma sample can be generated using the second mapping value based on the inverse mapping index.

[0383] In one embodiment, mapping values ​​based on bin indices from the minimum bin index to the maximum bin index can be derived based on the LMCS codeword. For example, the mapped predicted luma sample and the corrected restored luma sample can be generated based on the mapping values ​​based on the bin indices from the minimum bin index to the maximum bin index.

[0384] In one embodiment, the inverse mapping index may be derived based on a comparison between a mapping value based on the bin index from the minimum bin index to the maximum bin index and the value of the restored luma sample.

[0385] In one embodiment, the bin index includes a first bin index indicating a first mapping value, and the first bin index is derived based on a comparison between a mapping value based on the bin index from the minimum bin index to the maximum bin index and the value of the restored luma sample, and at least one of the corrected restored luma samples may be derived based on the restored luma sample and the first mapping value.

[0386] In one embodiment, the bin index includes a first bin index indicating a first mapping value, and the first bin index can be derived based on the comparison formula (lumaSample < LmcsPivot[idxYInv + 1]) included in Table 37. For example, in the formula, lumaSample represents the value of the target luma sample among the restored luma samples, idxYInv represents one of the bin indices, and LmcsPivot[idxYinv + 1] can represent one of the mapping values based on the bin index from the minimum bin index to the maximum bin index. At least one of the corrected restored luma samples may be derived based on the restored luma sample and the first mapping value. As an example, the bin index when the comparison formula (lumaSample < LmcsPivot[idxYInv + 1]) is true can be derived as the inverse mapping index.

[0387] In one embodiment, the comparison procedure (lumaSample < LmcsPivot[idxYInv + 1]) included in Table 37 can be performed for all of the bin indices from the minimum bin index to the maximum bin index.

[0388] In one embodiment, the image information includes residual information, a residual chroma sample is generated based on the residual information, an inverse scaling index is identified based on the LMCS-related information, a chroma residual scaling factor is derived based on the inverse scaling index, and a scaled residual chroma sample is generated based on the residual chroma sample and the chroma residual scaling factor.

[0389] In one embodiment, a resistive chroma sample is generated based on the resistive information, the bin index includes a first bin index pointing to a first mapping value, the first bin index is derived based on a comparison between the mapping values ​​based on the bin index from the minimum bin index to the maximum bin index and the values ​​of the restored chroma sample, a chroma resistive scaling factor is derived based on the first bin index, a scaled resistive chroma sample is generated based on the resistive chroma sample and the chroma resistive scaling factor, and a restored chroma sample may be generated based on the scaled resistive chroma sample.

[0390] In one embodiment, the image information includes information about linear LMCS, the information about LMCS data includes a linear LMCS flag indicating whether or not linear LMCS is applied, and the value of the linear LMCS flag may be 1 if the mapped predicted lumens sample is generated based on the information about linear LMCS.

[0391] In one embodiment, the information regarding the linear LMCS may include information regarding a first pivot point (e.g., P1 in Figure 12) and information regarding a second pivot point (e.g., P2 in Figure 12). For example, the input value and mapping value of the first pivot point are the minimum input value and minimum mapping value, respectively. The input value and mapping value of the second pivot point are the maximum input value and maximum mapping value, respectively. The input values ​​between the minimum input value and the maximum input value can be mapped linearly.

[0392] In one embodiment, the image information may include an SPS (sequence parameter set). The SPS may include a linear LMCS availability flag indicating whether linear LMCS is available.

[0393] In one embodiment, a minimum bin index (e.g., lmcs_min_bin_idx) and / or a maximum bin index (e.g., LmcsMaxBinIdx) can be derived based on the LMCS-related information. A first mapping value (LmcsPivot[lmcs_min_bin_idx]) can be derived based on the minimum bin index. A second mapping value (LmcsPivot[LmcsMaxBinIdx] or LmcsPivot[LmcsMaxBinIdx+1]) can be derived based on the maximum bin index. The value of the restored luma sample (e.g., lumaSample in Table 37 or Table 38) can be within the range from the first mapping value to the second mapping value. For example, the values ​​of all restored luma samples can be within the range from the first mapping value to the second mapping value. As another example, the values ​​of some of the restored luma samples can be within the range from the first mapping value to the second mapping value.

[0394] In one embodiment, the encoding device can generate a piece-wise index for chroma-residual scaling. The encoding device can derive a chroma-residual scaling factor based on the piece-wise index. The encoding device can generate a resistual chroma sample and a resistual chroma sample scaled based on the chroma-residual scaling factor.

[0395] In one embodiment, the chroma regi dual scaling factor is a single chroma regi dual scaling factor.

[0396] In one embodiment, the LMCS-related information may include an LMCS data field and information about a linear LMCS. The information about the linear LMCS may also be called information about a linear mapping. The LMCS data field may include a linear LMCS flag indicating whether the linear LMCS is applicable. If the value of the linear LMCS flag is 1, the mapped predictive LMCS sample can be generated based on the information about the linear LMCS.

[0397] In one embodiment, the image information may include information regarding the maximum input value and information regarding the maximum mapping value. The maximum input value is the same as the value of the information regarding the maximum input value (e.g., lmcs_max_input in Table 21). The maximum mapping value is the same as the value of the information regarding the maximum mapping value (e.g., lmcs_max_mapped in Table 21).

[0398] In one embodiment, the linear mapping information may include information regarding the input delta value of the second pivot point (e.g., lmcs_max_input_delta in Table 23) and information regarding the mapping delta value of the second pivot point (e.g., lmcs_max_mapped_delta in Table 23). The maximum input value can be derived based on the input delta value of the second pivot point, and the maximum mapping value can be derived based on the mapping delta value of the second pivot point.

[0399] In one embodiment, the maximum input value and the maximum mapping value can be derived based on at least one formula included in the aforementioned Table 24.

[0400] In one embodiment, the step of generating the mapped predicted luma sample may include the step of deriving a forward mapping scaling factor (e.g., the aforementioned ScaleCoeffSingle) for the predicted luma sample, and the step of generating the mapped predicted luma sample based on the forward mapping scaling factor. The forward mapping scaling factor is a single factor for the predicted luma sample.

[0401] In one embodiment, the forward mapping scaling factor can be derived based on at least one formula included in Tables 22 and / or 24 described above.

[0402] In one embodiment, the mapped predicted luma sample can be derived based on at least one formula included in Table 26 described above.

[0403] In one embodiment, the encoding device can derive an inverse mapping scaling factor (e.g., the aforementioned InvScaleCoeffSingle) for the restored luma sample (e.g., the aforementioned lumaSample). Also, the encoding device can generate a corrected restored luma sample (e.g., invSample) based on the restored luma sample and the inverse mapping scaling factor. The inverse mapping scaling factor is a single factor for the restored luma sample.

[0404] In one embodiment, the inverse mapping scaling factor can be derived using a piecewise index derived based on the restored luma sample.

[0405] In one embodiment, the piecewise index can be derived based on Table 36 described above. That is, the comparison procedure (lumaSample < LmcsPivot[idxYInv+1]) included in Table 37 can be repeatedly executed from the case where the piecewise index is the minimum bin index to the case where the piecewise index is the maximum bin index.

[0406] In one embodiment, the inverse mapping scaling factor can be derived based on at least one mathematical formula included in Table 19, Table 20, Table 22, Table 23 described above, or Mathematical Formula 11 or Mathematical Formula 12.

[0407] In one embodiment, the corrected restored luma sample can be derived based on Mathematical Formula 21 described above.

[0408] In one embodiment, the LMCS-related information may include information regarding the number of bins for deriving the mapped predicted luma samples (e.g., lmcs_num_bins_minus1 in Table 27). For example, the number of pivot points for luma mapping can be set to the same as the number of bins. In one example, the encoding device can generate the delta input values and delta mapping values of the pivot points for each of the number of bins. In one example, the input values and mapping values of the pivot points are derived based on the delta input values (e.g., lmcs_delta_input_cw[i] in Table 27) and the delta mapping values (e.g., lmcs_delta_mapped_cw[i] in Table 28), and the mapped predicted luma samples can be generated based on the input values (e.g., LmcsPivot_input[i] in Table 28, or InputPivot[i] in Table 10) and the mapping values (e.g., LmcsPivot_mapped[i] in Table 28, or LmcsPivot[i] in Table 10).

[0409] In one embodiment, the encoding device can derive the LMCS delta codewords based on at least one LMCS codeword included in the LMCS-related information and the original codeword (OrgCW), and can also derive the luma prediction samples mapped based on at least one LMCS codeword and the original codeword. In one example, the information regarding the linear mapping can include the information regarding the LMCS delta codewords.

[0410] In one embodiment, at least one LMCS codeword can be derived based on the sum of the LMCS delta codeword and OrgCW. For example, OrgCW is (1<<BitDepthY) / 16, where BitDepthY can indicate the luma bit depth. This embodiment can be performed based on Equation 12.

[0411] In one embodiment, the at least one LMCS codeword can be derived based on the addition of the LMCS delta codeword and OrgCW*(lmcs_max_bin_idx - lmcs_min_bin_idx + 1), where lmcs_max_bin_idx and lmcs_min_bin_idx are the maximum bin index and the minimum bin index respectively, and OrgCW is (1<<BitDepthY) / 16. This embodiment can be performed based on Formulas 15 and 16.

[0412] In one embodiment, the at least one LMCS codeword is a multiple of 2.

[0413] In one embodiment, when the luma bit depth (BitDepthY) of the restored luma sample is higher than 10, the at least one LMCS codeword is a multiple of 1<<(BitDepthY - 10).

[0414] In one embodiment, the at least one LMCS codeword is within the range from (OrgCW>>1) to (OrgCW<<1)-1.

[0415] FIG. 19 and FIG. 20 schematically show an example of an image / video decoding method and related components according to an embodiment of this document. The method disclosed in FIG. 19 can be performed by the decoding device disclosed in FIG. 3. Specifically, for example, S1900 in FIG. 19 can be performed by the entropy decoding unit 310 of the decoding device, S1910 and S1920 can be performed by the prediction unit 330 of the decoding device, S1930 can be performed by the residual processing unit 320 or the prediction unit 330 of the decoding device, and S1940 can be performed by the residual processing unit 320, the prediction unit 330, and / or the addition unit 340 of the decoding device. The method disclosed in FIG. 19 can include the embodiments described above in this document.

[0416] As shown in Figure 19, the decoding device can receive / acquire video / image information (S1900). The video / image information may include prediction mode information, LMCS-related information, and / or residual information. For example, the LMCS-related information may include information about the chroma mapping described above (e.g., forward mapping, inverse mapping, linear mapping), information about chroma residual scaling, and / or indices related to LMCS (or reshaping, reshaper) (e.g., maximum bin index, minimum bin index, mapping index). The decoding device can receive / acquire the image / video information via a bitstream.

[0417] The aforementioned image / video information may include various types of information according to the embodiments described herein. For example, the aforementioned image / video information may include information disclosed in at least one of the Tables 1 to 38 described above.

[0418] The decoding device can derive a prediction mode for the current block in the current picture based on the prediction mode information (S1910). The decoding device can derive at least one of the various modes disclosed in this document from among the interprediction modes.

[0419] The decoding device can generate a predicted luma sample (S1920). The decoding device can derive a predicted luma sample for the current block based on the prediction mode. The decoding device can generate a predicted luma sample by making a prediction for the original sample contained in the current block.

[0420] The decoding device can generate mapped predicted luma samples (S1930). The decoding device can generate mapped predicted luma samples based on a mapping procedure for luma samples. For example, the decoding device can derive input values ​​and mapping values ​​(output values) for pivot points for luma mapping, and can generate mapped predicted luma samples based on the input values ​​and mapping values. As an example, the decoding device can derive a (forward) mapping index (idxY) based on a first predicted luma sample, and can generate a first mapped predicted luma sample based on the input values ​​and mapping values ​​of the pivot points corresponding to the mapping index. As another example, linear mapping (linear reshaping, linear LMCS) may be used, and mapped predicted luma samples can be generated based on a forward mapping scaling factor derived from two pivot points in linear mapping, and thus the index derivation procedure can be omitted by linear mapping.

[0421] The decoding device can generate a residual chroma sample based on the residual information. For example, the decoding device can derive quantized transformation coefficients based on the residual information. The quantized transformation coefficients may have a one-dimensional vector form based on the coefficient scan order. The decoding device can derive transformation coefficients based on an inverse quantization procedure on the quantized transformation coefficients. The decoding device can derive a residual sample based on an inverse transformation procedure on the transformation coefficients. The residual sample may include a residual chroma sample and / or a residual chroma sample.

[0422] The decoding device can generate a reconstructed luma sample. The decoding device can generate a reconstructed luma sample based on the mapped predicted luma sample. Specifically, the decoding device can aggregate the aforementioned residual luma sample with the mapped predicted luma sample and generate a reconstructed luma sample based on the aggregate result.

[0423] The decoding device can generate a corrected restored luma sample. The decoding device can generate a corrected restored luma sample based on an inverse mapping procedure for the luma sample. The decoding device can generate a corrected restored luma sample based on information regarding the LMCS data and the restored luma sample (S1940). The decoding device can generate the corrected restored luma sample via an inverse mapping procedure for the restored luma sample.

[0424] The decoding device can generate scaled resistive chroma samples. Specifically, the decoding device can derive a chroma-residual scaling factor and generate scaled resistive chroma samples based on the chroma-residual scaling factor. Here, the chroma-residual scaling at the decoding end can also be called inverse chroma-residual scaling, inverse to that at the encoding end. Therefore, the chroma-residual scaling factor derived by the decoding device can be called the inverse chroma-residual scaling factor, and an inverse-scaled resistive chroma sample can be generated.

[0425] The decoding device can generate a reconstructed chroma sample. The decoding device can generate a reconstructed chroma sample based on a scaled resistive chroma sample. Specifically, the decoding device can perform a prediction procedure on the chroma component and generate a predicted chroma sample. The decoding device can generate a reconstructed chroma sample based on the sum of the predicted chroma sample and the scaled resistive chroma sample.

[0426] In one embodiment, mapping values ​​based on bin indices from the minimum bin index to the maximum bin index can be derived based on the LMCS codeword. For example, the mapped predicted luma sample and the corrected restored luma sample can be generated based on the mapping values ​​based on the bin indices from the minimum bin index to the maximum bin index.

[0427] In one embodiment, the LMCS-related information may include information about bins for mapping and inverse mapping, and information about an LMCS codeword. For example, a minimum bin index and a maximum bin index may be derived based on the information about the bins. A mapping value based on bin indices from the minimum bin index to the maximum bin index may be derived based on the information about the LMCS codeword. The mapping value may include a first mapping value (e.g., LmcsPivot[idxY]) and a second mapping value (e.g., LmcsPivot[idxYInv]). In the mapping procedure for the luma sample, a mapping index (e.g., idxY) may be derived based on the predicted luma sample, and the mapped predicted luma sample may be generated using the first mapping value based on the mapping index. In the inverse mapping procedure for the luma sample, an inverse mapping index may be derived based on the mapping value based on bin indices from the minimum bin index to the maximum bin index, and the modified restored luma sample may be generated using the second mapping value based on the inverse mapping index (e.g., idxYInv).

[0428] In one embodiment, the image information may include an SPS (sequence parameter set). The SPS may include a linear LMCS availability flag indicating whether linear LMCS is available.

[0429] In one embodiment, the bin index includes a first bin index that points to a first mapping value, the first bin index is derived based on a comparison of the mapping values ​​based on the bin index from the minimum bin index to the maximum bin index with the value of the restored luma sample, and at least one of the modified restored luma sample can be derived based on the restored luma sample and the first mapping value.

[0430] In one embodiment, the bin index includes a first bin index that points to a first mapping value, and the first bin index can be derived based on the comparison formula (lumaSample < LmcsPivot[idxYInv+1]) included in Table 37. For example, in the formula, lumaSample represents the value of a target luma sample among the restored luma samples, idxYInv represents one of the bin indices, and LmcsPivot[idxYinv+1] can represent one of the mapping values based on the bin indices from the minimum bin index to the maximum bin index. At least one of the corrected restored luma samples can be derived based on the restored luma sample and the first mapping value. As an example, the bin index when the comparison formula (lumaSample < LmcsPivot[idxYInv+1]) is true can be derived as the inverse mapping index.

[0431] In one embodiment, the comparison procedure (lumaSample < LmcsPivot[idxYInv+1]) included in Table 37 can be performed for all of the bin indices from the minimum bin index to the maximum bin index.

[0432] In one embodiment, the image information includes residual information, residual chroma samples are generated based on the residual information, an inverse scaling index is identified based on the LMCS related information, a chroma residual scaling factor is derived based on the inverse scaling index, and scaled residual chroma samples can be generated based on the residual chroma samples and the chroma residual scaling factor.

[0433] In one embodiment, a resistive chroma sample is generated based on the resistive information, the bin index includes a first bin index pointing to a first mapping value, the first bin index is derived based on a comparison between the mapping values ​​based on the bin index from the minimum bin index to the maximum bin index and the values ​​of the restored chroma sample, a chroma resistive scaling factor is derived based on the first bin index, a scaled resistive chroma sample is generated based on the resistive chroma sample and the chroma resistive scaling factor, and a restored chroma sample may be generated based on the scaled resistive chroma sample.

[0434] In one embodiment, the information regarding the LMCS data may include information regarding a linear LMCS. The information regarding the linear LMCS may also be referred to as information regarding a linear mapping. The information regarding the LMCS data may include a linear LMCS flag indicating whether or not a linear LMCS is applied. If the value of the linear LMCS flag is 1, the mapped predicted lumens can be generated based on the information regarding the linear LMCS.

[0435] In one embodiment, the information regarding the linear LMCS may include information regarding a first pivot point (e.g., P1 in Figure 12) and information regarding a second pivot point (e.g., P2 in Figure 12). For example, the input value and mapping value of the first pivot point are the minimum input value and minimum mapping value, respectively. The input value and mapping value of the second pivot point are the maximum input value and maximum mapping value, respectively. The input values ​​between the minimum input value and the maximum input value can be mapped linearly.

[0436] In one embodiment, a minimum bin index (e.g., lmcs_min_bin_idx) and / or a maximum bin index (e.g., LmcsMaxBinIdx) can be derived based on the LMCS-related information. A first mapping value (LmcsPivot[lmcs_min_bin_idx]) can be derived based on the minimum bin index. A second mapping value (LmcsPivot[LmcsMaxBinIdx] or LmcsPivot[LmcsMaxBinIdx+1]) can be derived based on the maximum bin index. The value of the restored luma sample (e.g., lumaSample in Table 37 or Table 38) is within the range from the first mapping value to the second mapping value. In one example, the values ​​of all restored luma samples are within the range from the first mapping value to the second mapping value. In another example, the values ​​of some of the restored luma samples are within the range from the first mapping value to the second mapping value.

[0437] In one embodiment, a piecewise index (e.g., idxYInv in Table 36, Table 37, or Table 38) can be identified based on the LMCS-related information. The decoding device can derive a chroma-residual scaling factor based on the piecewise index. The decoding device can generate a chroma-residual chroma sample scaled based on the chroma-residual scaling factor and the chroma-residual chroma sample.

[0438] In one embodiment, the chroma regi dual scaling factor may be a single chroma regi dual scaling factor.

[0439] In one embodiment, the image information may include information regarding the maximum input value and information regarding the maximum mapping value. The maximum input value is the same as the value of the information regarding the maximum input value (e.g., lmcs_max_input in Table 21). The maximum mapping value is the same as the value of the information regarding the maximum mapping value (e.g., lmcs_max_mapped in Table 21).

[0440] In one embodiment, the linear mapping information may include information regarding the input delta value of the second pivot point (e.g., lmcs_max_input_delta in Table 23) and information regarding the mapping delta value of the second pivot point (e.g., lmcs_max_mapped_delta in Table 23). The maximum input value can be derived based on the input delta value of the second pivot point, and the maximum mapping value can be derived based on the mapping delta value of the second pivot point.

[0441] In one embodiment, the maximum input value and the maximum mapping value can be derived based on at least one formula included in the aforementioned Table 24.

[0442] In one embodiment, the step of generating the mapped predicted luma sample may include the step of deriving a forward mapping scaling factor (e.g., the aforementioned ScaleCoeffSingle) for the predicted luma sample, and the step of generating the mapped predicted luma sample based on the forward mapping scaling factor. The forward mapping scaling factor is a single factor for the predicted luma sample.

[0443] In one embodiment, the inverse mapping scaling factor can be derived using a piecewise index derived based on the restored luma sample.

[0444] In one embodiment, the piecewise index can be derived based on Table 36 described above. That is, the comparison procedure (lumaSample < LmcsPivot[idxYInv + 1]) included in Table 37 can be repeatedly executed from the case where the piecewise index is the minimum bin index to the case where the piecewise index is the maximum bin index.

[0445] In one embodiment, the forward mapping scaling factor can be derived based on at least one mathematical formula included in Table 22 and / or Table 24 described above.

[0446] In one embodiment, the mapped predicted luma sample can be derived based on at least one mathematical formula included in Table 26 described above.

[0447] In one embodiment, the decoding device can derive an inverse mapping scaling factor (e.g., InvScaleCoeffSingle described above) for the restored luma sample (e.g., lumaSample described above). Further, the decoding device can generate a restored luma sample (e.g., invSample) corrected based on the restored luma sample and the inverse mapping scaling factor. The inverse mapping scaling factor is a single factor for the restored luma sample.

[0448] In one embodiment, the inverse mapping scaling factor can be derived based on at least one mathematical formula included in Table 19, Table 20, Table 22, Table 24 described above, or Formula 11 or Formula 12.

[0449] In one embodiment, the corrected restored luma sample can be derived based on Formula 21 described above.

[0450] In one embodiment, the LMCS-related information may include information regarding the number of bins for deriving the mapped predicted luma samples (e.g., lmcs_num_bins_minus1 in Table 27). For example, the number of pivot points for luma mapping can be set to the same as the number of bins. In one example, the decoding device can generate, for each of the number of bins, a delta input value and a delta mapping value of the pivot points. In one example, based on the delta input value (e.g., lmcs_delta_input_cw[i] in Table 27) and the delta mapping value (e.g., lmcs_delta_mapped_cw[i] in Table 28), the input value and the mapping value of the pivot points are derived, and based on the input value (e.g., LmcsPivot_input[i] in Table 28, or InputPivot[i] in Table 10) and the mapping value (e.g., LmcsPivot_mapped[i] in Table 28, or LmcsPivot[i] in Table 10), the mapped predicted luma samples can be generated.

[0451] In one embodiment, the decoding device can derive an LMCS delta codeword based on at least one LMCS codeword included in the LMCS-related information and an original codeword (OrgCW), and can also derive a luma prediction sample mapped based on at least one LMCS codeword and the original codeword. In one example, the information regarding the linear mapping can include information regarding the LMCS delta codeword.

[0452] In one embodiment, based on the addition of the LMCS delta codeword and OrgCW, at least one LMCS codeword can be derived. For example, OrgCW is (1<<BitDepthY) / 16, where BitDepthY can indicate the luma bit depth. This embodiment can be performed based on Equation 12.

[0453] In one embodiment, the at least one LMCS codeword can be derived based on the sum of the LMCS delta codeword and OrgCW * (lmcs_max_bin_idx - lmcs_min_bin_idx + 1), where lmcs_max_bin_idx and lmcs_min_bin_idx are the maximum bin index and the minimum bin index, respectively, and OrgCW is (1 << BitDepthY) / 16. This embodiment can be performed based on Equations 15 and 16.

[0454] In one embodiment, the at least one LMCS codeword can be a multiple of 2.

[0455] In one embodiment, when the luma bit depth (BitDepthY) of the restored luma sample is higher than 10, the at least one LMCS codeword is a multiple of 1 << (BitDepthY - 10).

[0456] In one embodiment, the at least one LMCS codeword is within the range from (OrgCW >> 1) to (OrgCW << 1) - 1.

[0457] In the above-described embodiments, the method is described based on a flowchart as a series of steps or blocks, but the corresponding embodiments are not limited to the order of the steps. A certain step can occur in a different order from the steps described above, or simultaneously. Also, those skilled in the art can understand that the steps shown in the flowchart are not exclusive, and different steps may be included, or one or more steps of the flowchart can be deleted without affecting the scope of the embodiments of this document.

[0458] The method according to the embodiments of this document described above can be realized in the form of software, and the encoding device and / or decoding device according to this document can be included in a device that performs image processing, such as a TV, a computer, a smartphone, a set-top box, a display device, etc.

[0459] In this document, when embodiments are implemented in software, the methods described above can be implemented by modules (processes, functions, etc.) that perform the functions described above. These modules are stored in memory and can be executed by a processor. The memory may be internal or external to the processor and may be connected to the processor by various well-known means. The processor may include an ASIC (application-specific integrated circuit), other chipsets, logic circuits, and / or data processing devices. The memory may include ROM (read-only memory), RAM (random access memory), flash memory, memory cards, storage media, and / or other storage devices. That is, the embodiments described in this document may be implemented on a processor, microprocessor, controller, or chip. For example, the functional units shown in each drawing may be implemented on a computer, processor, microprocessor, controller, or chip. In this case, information on instructions or algorithms for implementation may be stored on a digital storage medium.

[0460] Furthermore, the decoding and encoding devices to which the embodiments of this document apply may include multimedia broadcasting transceivers, mobile communication terminals, home cinema video equipment, digital cinema video equipment, surveillance cameras, video interaction devices, real-time communication devices such as video communication, mobile streaming devices, storage media, camcorders, customized video (VoD) service providers, OTT video (Over the top video) devices, internet streaming service providers, 3D video devices, VR (virtual reality) devices, AR (argumente reality) devices, image phone video devices, transportation terminals (e.g., vehicle terminals (including autonomous vehicles), airplane terminals, ship terminals, etc.), and medical video equipment, and may be used to process video signals or data signals. For example, OTT video (Over the top video) devices may include game consoles, Blu-ray players, internet access TVs, home theater systems, smartphones, tablet PCs, DVRs (Digital Video Recorders), etc.

[0461] Furthermore, the processing methods to which the embodiments of this document apply can be produced in the form of programs executed on a computer and stored on a computer-readable recording medium. Multimedia data having a data structure according to the embodiments of this document can also be stored on a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distributed storage devices that store data to be read by a computer. The computer-readable recording medium may include, for example, Blu-ray discs (BDs), general-purpose serial buses (USBs), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer-readable recording medium also includes media implemented in the form of carrier waves (e.g., transmission over the Internet). Furthermore, a bitstream generated by an encoding method can be stored on a computer-readable recording medium or transmitted over a wired wireless network.

[0462] Furthermore, the embodiments described herein can be implemented as computer program products using program code, and the program code can be executed on a computer according to the embodiments described herein. The program code can be stored on a computer-readable carrier.

[0463] Figure 21 shows an example of a content streaming system to which the embodiments disclosed in this document may be applied.

[0464] Referring to Figure 21, the content streaming system to which the embodiments described in this document apply can broadly include an encoding server, a streaming server, a web server, media storage, user equipment, and multimedia input devices.

[0465] The encoding server is responsible for compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and then transmitting this bitstream to the streaming server. As an alternative, if a multimedia input device such as a smartphone, camera, or camcorder directly generates the bitstream, the encoding server may be omitted.

[0466] The bitstream can be generated by an encoding method or a bitstream generation method to which an embodiment of this document applies, and the streaming server can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0467] The streaming server transmits multimedia data to user devices based on user requests via a web server, and the web server acts as an intermediary to inform users about available services. When a user requests a desired service from the web server, the web server transmits this to the streaming server, and the streaming server transmits multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server controls the commands and responses between the devices within the content streaming system.

[0468] The streaming server can receive content from a media storage and / or encoding server. For example, if it starts receiving content from the encoding server, it can receive the content in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0469] Examples of user devices include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (such as smartwatches, smart glasses, and HMDs), digital TVs, desktop computers, and digital signage.

[0470] Each server within the aforementioned content streaming system can be operated as a distributed server, in which case the data received by each server can be processed in a distributed manner.

[0471] The claims described herein can be combined in various ways. For example, the technical features of the method claims herein can be combined to realize an apparatus, and the technical features of the apparatus claims herein can be combined to realize a method. Furthermore, the technical features of the method claims and the technical features of the apparatus claims herein can be combined to realize an apparatus, and the technical features of the method claims and the technical features of the apparatus claims herein can be combined to realize a method.

Claims

1. In an image decoding method performed by a decoding device, The steps include: obtaining image information from a bitstream, including prediction mode information and LMCS (luma mapping with chroma scaling) related information; The steps include: deriving an inter-prediction mode for the current block in the current picture based on the aforementioned prediction mode information; The steps include: deriving motion information for the current block based on the inter prediction mode; The steps include generating a predicted luma sample for the current block based on the motion information for the current block, A step of generating mapped predicted luma samples based on the mapping procedure for the predicted luma samples, The steps include generating a corrected restored luma sample based on an inverse mapping procedure for the luma sample, The aforementioned LMCS-related information includes information regarding bins for LMCS, Based on the information relating to the bins, the minimum bin index and the maximum bin index are derived. The mapping value indexed by 0 is equal to 0. The mapping values ​​indexed by indices 1 to 16 are derived based on the LMCS codewords. In the mapping procedure for the predicted luma sample, The mapping index is derived based on the predicted luma sample, and, The mapped predicted luma samples are generated by using the mapping values ​​indexed by the mapping index. In the in-mapping procedure for the Luma sample, Based on the mapping value indexed by the bin indices from the minimum bin index to the maximum bin index, an inverse mapping index is derived, and By using the mapping values ​​indexed by the inverse mapping index, the corrected restored luma sample is generated. To derive the inverse mapping index, the inverse mapping procedure for the Luma sample is: The steps include determining whether the mapping value based on the case where the inverse mapping index is equal to the minimum bin index is greater than the value of the restored luma sample, The steps include determining whether the mapping value based on the case where the inverse mapping index is equal to the maximum bin index minus 1 is greater than the value of the restored luma sample, The step includes determining whether the mapping value based on the case where the inverse mapping index is equal to the maximum bin index is greater than the value of the restored luma sample, based on the determination that the mapping value based on the case where the inverse mapping index is equal to the maximum bin index is not greater than the value of the restored luma sample, A method in which the inverse mapping index is derived as the maximum bin index based on the determination that the mapping value is greater than the value of the restored luma sample, in the case in the case in the case in the case in the maximum bin index is equal to the maximum bin index.

2. In an image encoding method performed by an encoding device, Steps to derive the interpretation mode, The steps include generating motion information for the current block based on the aforementioned inter-prediction mode, The steps include generating a predicted luma sample for the current block based on the motion information for the current block, A step of generating mapped predicted luma samples based on the mapping procedure for the predicted luma samples, The steps include generating LMCS (luma mapping with chroma scaling) related information based on the mapped predicted luma samples, A step to generate a corrected restored luma sample based on an inverse mapping procedure for the luma sample, The step includes encoding image information including the LMCS-related information, The aforementioned LMCS-related information includes information regarding bins for LMCS, Based on the information relating to the bins, the minimum bin index and the maximum bin index are derived. The mapping value indexed by 0 is equal to 0. The mapping values ​​indexed by indices 1 to 16 are derived based on the LMCS codewords. In the mapping procedure for the predicted luma sample, The mapping index is derived based on the predicted luma sample, and, The mapped predicted luma samples are generated by using the mapping values ​​indexed by the mapping index. In the in-mapping procedure for the Luma sample, Based on the mapping value indexed by the bin indices from the minimum bin index to the maximum bin index, an inverse mapping index is derived, and By using the mapping values ​​indexed by the inverse mapping index, the corrected restored luma sample is generated. To derive the inverse mapping index, the inverse mapping procedure for the Luma sample is: The steps include determining whether the mapping value based on the case where the inverse mapping index is equal to the minimum bin index is greater than the value of the restored luma sample, The steps include determining whether the mapping value based on the case where the inverse mapping index is equal to the maximum bin index minus 1 is greater than the value of the restored luma sample, The step includes determining whether the mapping value based on the case where the inverse mapping index is equal to the maximum bin index is greater than the value of the restored luma sample, based on the determination that the mapping value based on the case where the inverse mapping index is equal to the maximum bin index is not greater than the value of the restored luma sample, A method in which the inverse mapping index is derived as the maximum bin index based on the determination that the mapping value is greater than the value of the restored luma sample, in the case in the case in the case in the case in the maximum bin index is equal to the maximum bin index.

3. In a method for transmitting data related to image information, A step of acquiring a bitstream of image information including LMCS (luma mapping with chroma scaling) related information and residual information, The aforementioned LMCS-related information is, Steps to derive the interpretation mode, The steps include generating motion information for the current block based on the aforementioned inter-prediction mode, The steps include generating a predicted luma sample for the current block based on the motion information for the current block, A step of generating mapped predicted luma samples based on the mapping procedure for the predicted luma samples, The steps include generating LMCS-related information based on the mapped predicted luma samples, and generating the following: The residual information is generated by a step of generating a corrected restored luma sample based on an inverse mapping procedure for the luma sample, and The step of transmitting the data, which includes the bitstream of the image information including the LMCS-related information, The aforementioned image information further includes information regarding the bins for LMCS, Based on the information relating to the bins, the minimum bin index and the maximum bin index are derived. The mapping value indexed by 0 is equal to 0. The mapping values ​​indexed by indices 1 to 16 are derived based on the LMCS codewords. In the mapping procedure for the predicted luma sample, The mapping index is derived based on the predicted luma sample, and, The mapped predicted luma samples are generated by using the mapping values ​​indexed by the mapping index. In the in-mapping procedure for the Luma sample, Based on the mapping value indexed by the bin indices from the minimum bin index to the maximum bin index, an inverse mapping index is derived, and By using the mapping values ​​indexed by the inverse mapping index, the corrected restored luma sample is generated. To derive the inverse mapping index, the inverse mapping procedure for the Luma sample is: The steps include determining whether the mapping value based on the case where the inverse mapping index is equal to the minimum bin index is greater than the value of the restored luma sample, The steps include determining whether the mapping value based on the case where the inverse mapping index is equal to the maximum bin index minus 1 is greater than the value of the restored luma sample, The step includes determining whether the mapping value based on the case where the inverse mapping index is equal to the maximum bin index is greater than the value of the restored luma sample, based on the determination that the mapping value based on the case where the inverse mapping index is equal to the maximum bin index is not greater than the value of the restored luma sample, A method in which the inverse mapping index is derived as the maximum bin index based on the determination that the mapping value is greater than the value of the restored luma sample, in the case in the case in the case in the case in the maximum bin index is equal to the maximum bin index.

Images