Aps signaling-based video or image coding

Hierarchical signaling of ALF and LMCS information through APS IDs in video/image coding systems improves compression efficiency and visual quality for high-resolution video, addressing the challenges of high-demand media formats like VR and AR.

KR103045784B1Active Publication Date: 2026-11-06LG ELECTRONICS INC
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
KR1020257015475
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-02-28
Filing Date
2020-02-25
Publication Date
2026-11-06
Estimated Expiration
2040-02-25

AI Technical Summary

Technical Problem

The increasing demand for high-resolution and high-quality video, including immersive media like VR and AR, necessitates high-efficiency image/video compression technologies to reduce transmission and storage costs while maintaining visual quality.

Method used

Hierarchical signaling of ALF-related and LMCS-related information through APS IDs in picture or slice headers, allowing adaptive application of these technologies at the picture, slice, and coding block levels.

Benefits of technology

Enhances overall image/video compression efficiency and improves subjective/objective visual quality by enabling efficient filtering and signaling of ALF and LMCS data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112025052605605-PAT00074_ABST
    Figure 112025052605605-PAT00074_ABST
Patent Text Reader

Abstract

According to the disclosure of this document, ALF parameters and / or LMCS parameters can be signaled hierarchically, thereby reducing the amount of data that needs to be signaled for video / image coding and increasing coding efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] This technology relates to APS signaling-based video or image coding. Background Technology

[0002] Recently, the demand for high-resolution, high-quality video, such as 4K or 8K or higher UHD (Ultra High Definition) video, is increasing across various fields. As video data becomes higher resolution and higher quality, the amount of information or bits transmitted increases relative to existing video data; therefore, when transmitting video data using media such as existing wired or wireless broadband lines or storing video data using existing storage media, transmission and storage costs increase.

[0003] In addition, interest in and demand for immersive media such as VR (Virtual Reality), AR (Artificial Reality) content, and holograms have recently been increasing, and the broadcasting of video content with characteristics different from reality, such as game footage, is on the rise.

[0004] Accordingly, high-efficiency image / video compression technology is required to effectively compress, transmit, store, and play back high-resolution, high-quality image / video information having various characteristics as described above.

[0005] In addition, there is discussion regarding technologies such as LMCS (luma mapping with chroma scaling) and ALF (adaptive loop filtering) to improve compression efficiency and enhance subjective and objective visual quality. To efficiently apply these technologies, a method for efficiently signaling related information is required. means of solving the problem

[0006] According to one embodiment of the present document, a method and apparatus for increasing image / video coding efficiency are provided.

[0007] According to one embodiment of the present document, a method and apparatus for hierarchically signaling ALF-related information and / or LMCS-related information are provided.

[0008] According to one embodiment of the present document, ALF data and / or LMCS data may be signaled conditionally through header information (picture header or slice header).

[0009] According to one embodiment of the present document, ALF data and / or LMCS data may be signaled through an APS, and APS ID information indicating the ID of the referenced APS may be signaled through header information (picture header or slice header).

[0010] According to one embodiment of the present document, APS ID information referenced for ALF data and APS ID information referenced for LMCS data may be signaled separately.

[0011] According to one embodiment of the present document, a video / image decoding method performed by a decoding device is provided.

[0012] According to one embodiment of the present document, a decoding device that performs video / image decoding is provided.

[0013] According to one embodiment of the present document, a video / image encoding method performed by an encoding device is provided.

[0014] According to one embodiment of the present document, an encoding device for performing video / image encoding is provided.

[0015] According to one embodiment of the present document, a computer-readable digital storage medium is provided that stores encoded video / image information generated according to a video / image encoding method disclosed in at least one of the embodiments of the present document.

[0016] According to one embodiment of the present document, a computer-readable digital storage medium is provided that stores encoded information or encoded video / image information, which causes a video / image decoding method disclosed in at least one of the embodiments of the present document to be performed by a decoding device. Effects of the invention

[0017] According to one embodiment of the present document, overall image / video compression efficiency can be increased.

[0018] According to one embodiment of the present document, subjective / objective visual quality can be improved through efficient filtering.

[0019] According to one embodiment of the present document, ALF and / or LMCS can be adaptedly applied in units of pictures, slices and / or coding blocks.

[0020] According to one embodiment of the present document, ALF-related information can be efficiently signaled.

[0021] According to one embodiment of the present document, LMCS-related information can be efficiently signaled. Brief explanation of the drawing

[0022] Figure 1 schematically illustrates an example of a video / image coding system to which embodiments of the present document can be applied. FIG. 2 is a diagram schematically illustrating the configuration of a video / image encoding device to which embodiments of the present document can be applied. FIG. 3 is a diagram schematically illustrating the configuration of a video / image decoding device to which embodiments of the present document can be applied. Figure 4 illustrates an exemplary hierarchical structure for a coded image / video. Figure 5 is a flowchart schematically illustrating an example of an ALF procedure. Figures 6a and 6b show examples of ALF filter shapes. Figure 7 shows an example of the hierarchical structure of ALF data. Figure 8 shows another example of the hierarchical structure of ALF data. FIG. 9 illustrates an exemplary hierarchical structure of a CVS according to one embodiment of the present document. FIG. 10 illustrates an exemplary LMCS structure according to one embodiment of the present document. FIG. 11 illustrates an LMCS structure according to another embodiment of the present document. Figure 12 shows a graph representing an exemplary forward mapping. FIGS. 13 and 14 schematically illustrate an example of a video / image encoding method and related components according to the embodiment(s) of the present document. FIGS. 15 and 16 schematically illustrate an example of an image / video decoding method and related components according to an embodiment of the present document. FIG. 17 shows an example of a content streaming system to which the embodiments disclosed in this document can be applied. Specific details for implementing the invention

[0023] As the disclosure of this document is subject to various modifications and may have various embodiments, specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the disclosure to specific embodiments. The terms used in this document are used merely to describe specific embodiments and are not intended to limit the technical scope of the embodiments described in this document. Singular expressions include plural expressions unless the context clearly indicates otherwise. Terms such as "comprising" or "having" in this document are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the document, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0024] Meanwhile, each component in the drawings described in this document is depicted independently for the convenience of describing different characteristic functions and does not imply that each component is implemented in separate hardware or separate software. For example, two or more components may be combined to form a single component, or a single component may be divided into multiple components. Embodiments in which each component is integrated and / or separated are also included within the scope of this document.

[0025] Hereinafter, embodiments of the present document will be described with reference to the attached drawings. In the following, the same reference numerals may be used for identical components in the drawings, and redundant descriptions of identical components may be omitted.

[0026] Figure 1 schematically illustrates an example of a video / image coding system to which embodiments of the present document can be applied.

[0027] Referring to FIG. 1, a video / image coding system may include a first device (source device) and a second device (receiving device). The source device may transmit encoded video / image information or data to the receiving device in the form of a file or streaming via a digital storage medium or a network.

[0028] The source device may include a video source, an encoding device, and a transmission unit. The receiving device may include a receiving unit, a decoding device, and a renderer. The encoding device may be called a video / image encoding device, and the decoding device may be called a video / image decoding device. A transmitter may be included in the encoding device. A receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be composed of a separate device or an external component.

[0029] A video source may acquire video / images through processes such as video / image capture, synthesis, or generation. The video source may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / images, etc. The video / image generation device may include, for example, a computer, a tablet, and a smartphone, etc., and may generate video / images (electronically). For example, virtual video / images may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process in which related data is generated.

[0030] The encoding device can encode input video. The encoding device can perform a series of procedures, such as prediction, transformation, and quantization, for compression and coding efficiency. The encoded data (encoded image / video information) can be output in the form of a bitstream.

[0031] The transmission unit can transmit encoded video information or data output in the form of a bitstream to the receiving unit of a receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmission unit may include elements for creating a media file through a predetermined file format and elements for transmission via a broadcasting / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0032] The decoding device can decode video / images by performing a series of procedures such as inverse quantization, inverse transform, and prediction corresponding to the operation of the encoding device.

[0033] The renderer can render the decoded video / image. The rendered video / image can be displayed through the display unit.

[0034] This document relates to video / video coding. For example, the methods / exemplars disclosed in this document may be applied to methods disclosed in the VVC (versatile video coding) standard. Additionally, the methods / exemplars disclosed in this document may be applied to methods disclosed in the EVC (essential video coding) standard, AV1 (AOMedia Video 1) standard, AVS2 (2nd generation of audio video coding standard), or next-generation video / video coding standards (e.g., H.267 or H.268).

[0035] This document presents various embodiments regarding video / image coding, and unless otherwise noted, the embodiments may be performed in combination with one another.

[0036] In this document, "video" may refer to a set of images over time. "Picture" generally refers to a unit representing a single image at a specific time, and "slice" or "tile" are units that constitute a part of a picture in coding. A slice or tile may include one or more CTUs (coding tree units). A single picture may be composed of one or more slices or tiles. A tile is a rectangular region of CTUs within a particular tile column and a particular tile row in a picture. The tile column is a rectangular region of CTUs having a height equal to the height of the picture and a width specified by syntax elements in the picture parameter set. The tile row is a rectangular region of CTUs, and the rectangular region has a width specified by syntax elements in the picture parameter set and a width equal to the width of the picture.A tile scan may represent a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in a CTU raster scan in a tile, and tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. A slice may include an integer number of complete tiles or an integer number of consecutive complete CTU rows within a tile of a picture that may be exclusively contained in a single NAL unit.

[0037] Meanwhile, a picture can be divided into two or more subpictures. A subpicture can be a rectangular region of one or more slices within a picture.

[0038] A pixel or pel can refer to the smallest unit that constitutes a picture (or image). Additionally, the term 'sample' may be used as a counterpart to pixel. Generally, a sample can represent a pixel or its value, and it may represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chroma component.

[0039] A unit may represent a basic unit of image processing. A unit may include at least one of a specific area of ​​a picture and information related to that area. A unit may include one luminance block and two chroma (e.g., cb, cr) blocks. Depending on the case, the term unit may be used interchangeably with terms such as block or area. In general, an MxN block may include samples (or sample arrays) or a set (or array) of transform coefficients consisting of M columns and N rows.

[0040] In this document, "A or B" may mean "only A," "only B," or "both A and B." Alternatively, in this document, "A or B" may be interpreted as "A and / or B." For example, in this document, "A, B or C" may mean "only A," "only B," "only C," or "any combination of A, B and C."

[0041] The slash ( / ) or comma used in this document may mean "and / or." For example, "A / B" may mean "A and / or B." Accordingly, "A / B" may mean "only A," "only B," or "both A and B." For example, "A, B, C" may mean "A, B or C."

[0042] In this document, "at least one of A and B" may mean "only A," "only B," or "both A and B." Additionally, in this document, the expressions "at least one of A or B" or "at least one of A and / or B" may be interpreted as synonymous with "at least one of A and B."

[0043] Additionally, in this document, "at least one of A, B and C" may mean "only A," "only B," "only C," or "any combination of A, B and C." Also, "at least one of A, B or C" or "at least one of A, B and / or C" may mean "at least one of A, B and C."

[0044] Additionally, parentheses used in this document may mean "for example." Specifically, where indicated as "prediction (intra-prediction)," "intra-prediction" may be proposed as an example of "prediction." In other words, "prediction" in this document is not limited to "intra-prediction," and "intra-prediction" may be proposed as an example of "prediction." Furthermore, even when indicated as "prediction (i.e., intra-prediction)," "intra-prediction" may be proposed as an example of "prediction."

[0045] Technical features described individually within a single drawing in this document may be implemented individually or simultaneously.

[0046] FIG. 2 is a diagram schematically illustrating the configuration of a video / image encoding device to which embodiments of the present document may be applied. Hereinafter, the term "encoding device" may include an image encoding device and / or a video encoding device.

[0047] Referring to FIG. 2, the encoding device (200) may be configured to include an image partitioner (210), a predictor (220), a residual processor (230), an entropy encoder (240), an adder (250), a filter (260), and a memory (270). The predictor (220) may include an inter-predictor (221) and an intra-predictor (222). The residual processor (230) may include a transformer (232), a quantizer (233), a dequantizer (234), and an inverse transformer (235). The residual processor (230) may further include a subtractor (231). The addition unit (250) may be referred to as a reconstructor or a reconstructed block generator. The above-described image segmentation unit (210), prediction unit (220), residual processing unit (230), entropy encoding unit (240), addition unit (250), and filtering unit (260) may be configured by one or more hardware components (e.g., an encoder chipset or processor) according to the embodiment. Additionally, the memory (270) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (270) as an internal / external component.

[0048] The image segmentation unit (210) can divide an input image (or picture, frame) input to an encoding device (200) into one or more processing units. For example, the processing unit may be called a coding unit (CU). In this case, the coding unit may be recursively divided from a coding tree unit (CTU) or a largest coding unit (LCU) according to a QTBTTT (Quad-tree binary-tree ternary-tree) structure. For example, one coding unit may be divided into multiple coding units of a deeper depth based on a quad-tree structure, a binary-tree structure, and / or a ternary structure. In this case, for example, the quad-tree structure may be applied first and the binary-tree structure and / or ternary structure may be applied later. Or the binary-tree structure may be applied first. A coding procedure according to this document may be performed based on the final coding unit that is no longer divided. In this case, based on coding efficiency according to image characteristics, the maximum coding unit may be used directly as the final coding unit, or, if necessary, the coding unit may be recursively divided into lower-depth coding units so that a coding unit of the optimal size is used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration described later. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may each be divided or partitioned from the final coding unit described above.The above prediction unit may be a unit of sample prediction, and the above transformation unit may be a unit that derives transformation coefficients and / or a unit that derives a residual signal from transformation coefficients.

[0049] The term "unit" may be used interchangeably with terms such as "block" or "area" depending on the context. In general, an MxN block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chroma component. A sample may be used to refer to a single picture (or image) as a term corresponding to a pixel or pel.

[0050] The encoding device (200) can generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (predicted block, prediction sample array) output from an inter prediction unit (221) or an intra prediction unit (222) from an input image signal (original block, original sample array), and the generated residual signal is transmitted to a conversion unit (232). In this case, as illustrated, the unit that subtracts the prediction signal (predicted block, prediction sample array) from the input image signal (original block, original sample array) within the encoder (200) may be called a subtraction unit (231). The prediction unit performs a prediction for a block to be processed (hereinafter referred to as the current block) and can generate a predicted block containing prediction samples for the current block. The prediction unit can determine whether intra prediction is applied or inter prediction is applied at the current block or CU level. The prediction unit can generate various information regarding prediction, such as prediction mode information, as described below in the description of each prediction mode, and transmit it to the entropy encoding unit (240). The information regarding prediction can be encoded in the entropy encoding unit (240) and output in the form of a bitstream.

[0051] The intra prediction unit (222) can predict the current block by referring to samples within the current picture. The referenced samples may be located near the current block or away from it, depending on the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a Planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is merely an example, and depending on the settings, more or fewer directional prediction modes may be used. The intra prediction unit (222) may also determine the prediction mode applied to the current block by using the prediction mode applied to the surrounding blocks.

[0052] The inter prediction unit (221) can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, to reduce the amount of motion information transmitted in the inter prediction mode, motion information can be predicted in blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, neighboring blocks may include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. The reference picture containing the reference blocks and the reference picture containing the temporal neighboring blocks may be the same or different. The above temporal surrounding blocks may be referred to by names such as collocated reference block, collocated CU (colCU), etc., and the reference picture containing the above temporal surrounding blocks may be referred to as a collocated picture (colPic). For example, the inter prediction unit (221) may construct a list of motion information candidates based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, for example, in the case of skip mode and merge mode, the inter prediction unit (221) may use the motion information of surrounding blocks as motion information of the current block. In the case of skip mode, unlike merge mode, a residual signal may not be transmitted.In the motion vector prediction (MVP) mode, the motion vector of surrounding blocks is used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0053] The prediction unit (220) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit may apply intra prediction or inter prediction for a single block, and may also apply intra prediction and inter prediction simultaneously. This may be called combined inter and intra prediction (CIIP). Additionally, the prediction unit may be based on an intra block copy (IBC) prediction mode or a palette mode for predicting a block. The IBC prediction mode or palette mode may be used for content video / video coding, such as in games, for example, screen content coding (SCC). IBC basically performs prediction within the current picture, but it can be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC may use at least one of the inter prediction techniques described in this document. The palette mode can be viewed as an example of intra coding or intra prediction. When palette mode is applied, sample values ​​within the picture can be signaled based on information regarding the palette table and palette index.

[0054] The prediction signal generated through the prediction unit (including the inter prediction unit (221) and / or the intra prediction unit (222)) may be used to generate a restored signal or to generate a residual signal. The transformation unit (232) may generate transform coefficients by applying a transformation technique to the residual signal. For example, the transformation technique may include at least one of a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), a Graph-Based Transform (GBT), or a Conditionally Non-linear Transform (CNT). Here, GBT refers to a transformation obtained from a graph when the relationship information between pixels is represented as a graph. CNT refers to a transformation obtained based on generating a prediction signal using all previously reconstructed pixels. Additionally, the transformation process may be applied to a pixel block of the same size in a square, or to a block of variable size that is not square.

[0055] The quantization unit (233) quantizes the transformation coefficients and transmits them to the entropy encoding unit (240), and the entropy encoding unit (240) can encode the quantized signal (information regarding the quantized transformation coefficients) and output it as a bitstream. The information regarding the quantized transformation coefficients may be called residual information. The quantization unit (233) can rearrange the block-shaped quantized transformation coefficients into a one-dimensional vector form based on the coefficient scan order, and can also generate information regarding the quantized transformation coefficients based on the one-dimensional vector-shaped quantized transformation coefficients. The entropy encoding unit (240) can perform various encoding methods such as, for example, exponential Golomb, CAVLC (context-adaptive variable length coding), CABAC (context-adaptive binary arithmetic coding), etc. The entropy encoding unit (240) may encode information necessary for video / image restoration (e.g., values ​​of syntax elements) together or separately, in addition to the quantized transform coefficients. The encoded information (e.g., encoded image / video information) may be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The image / video information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Additionally, the image / video information may further include general constraint information. In this document, information and / or syntax elements transmitted / signaled from the encoding device to the decoding device may be included in the image / video information. The image / video information may be encoded through the encoding procedure described above and included in the bitstream.The above bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmission unit (not shown) that transmits the signal output from the entropy encoding unit (240) and / or a storage unit (not shown) that stores it may be configured as internal / external elements of the encoding device (200), or the transmission unit may be included in the entropy encoding unit (240).

[0056] Quantized transformation coefficients output from the quantization unit (233) can be used to generate a prediction signal. For example, a residual signal (residual block or residual samples) can be restored by applying inverse quantization and inverse transformation to the quantized transformation coefficients through the inverse quantization unit (234) and the inverse transformation unit (235). An adder (250) can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the restored residual signal to the prediction signal output from the inter-prediction unit (221) or the intra-prediction unit (222). In cases where there is no residual for the block to be processed, such as when a skip mode is applied, the predicted block can be used as the reconstructed block. The adder (250) may be called a reconstruction unit or a reconstruction block generation unit. The generated restoration signal can be used for intra prediction of the next processing target block within the current picture, and can also be used for inter prediction of the next picture after filtering as described below.

[0057] Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture encoding and / or restoration process.

[0058] The filtering unit (260) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (260) can generate a modified restored picture by applying various filtering methods to the restored picture, and can store the modified restored picture in memory (270), specifically in the DPB of memory (270). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit (260) can generate various information regarding filtering and transmit it to the entropy encoding unit (240), as described below in the description of each filtering method. The information regarding filtering can be encoded in the entropy encoding unit (240) and output in the form of a bitstream.

[0059] The modified restored picture transmitted to the memory (270) can be used as a reference picture in the inter-prediction unit (221). Through this, when inter-prediction is applied, the encoding device can avoid prediction mismatches between the encoding device (200) and the decoding device, and can also improve encoding efficiency.

[0060] The memory (270) DPB can store the modified restored picture to be used as a reference picture in the inter-prediction unit (221). The memory (270) can store motion information of blocks from which motion information is derived (or encoded) within the current picture and / or motion information of blocks within the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit (221) to be used as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. The memory (270) can store restoration samples of the blocks restored within the current picture and transmit them to the intra-prediction unit (222).

[0061] FIG. 3 is a diagram schematically illustrating the configuration of a video / image decoding device to which embodiments of the present document may be applied. The term "decoding device" below may include an image decoding device and / or a video decoding device.

[0062] Referring to FIG. 3, the decoding device (300) may be configured to include an entropy decoder (310), a residual processor (320), a predictor (330), an adder (340), a filter (350), and a memory (360). The predictor (330) may include an inter-predictor (331) and an intra-predictor (332). The residual processor (320) may include a dequantizer (321) and an inverse transformer (321). The aforementioned entropy decoding unit (310), residual processing unit (320), prediction unit (330), addition unit (340), and filtering unit (350) may be configured by a single hardware component (e.g., a decoder chipset or a processor) according to an embodiment. Additionally, the memory (360) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (360) as an internal / external component.

[0063] When a bitstream containing image / video information is input, the decoding device (300) can restore the image in correspondence with the process in which the image / video information is processed by the encoding device of FIG. 2. For example, the decoding device (300) can derive units / blocks based on block division information obtained from the bitstream. The decoding device (300) can perform decoding using a processing unit applied by the encoding device. Accordingly, the processing unit for decoding may be, for example, a coding unit, and the coding unit may be divided according to a quad tree structure, a binary tree structure, and / or a binary tree structure from a coding tree unit or a maximum coding unit. One or more conversion units may be derived from the coding unit. And, the restored image signal decoded and output through the decoding device (300) can be played back through a playback device.

[0064] The decoding device (300) can receive a signal output from the encoding device of FIG. 2 in the form of a bitstream, and the received signal can be decoded through an entropy decoding unit (310). For example, the entropy decoding unit (310) can parse the bitstream to derive information (e.g., image / video information) necessary for image restoration (or picture restoration). The image / video information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Additionally, the image / video information may further include general constraint information. The decoding device can decode the picture based on information regarding the parameter sets and / or the general constraint information. The signaling / receiving information and / or syntax elements described below in this document can be obtained from the bitstream by being decoded through the decoding procedure. For example, the entropy decoding unit (310) can decode information within a bitstream based on coding methods such as exponential chord coding, CAVLC, or CABAC, and output values ​​of syntax elements required for image restoration and quantized values ​​of transformation coefficients regarding residuals. More specifically, the CABAC entropy decoding method can receive a bin corresponding to each syntax element in a bitstream, determine a context model using information of the syntax element to be decoded and decoding information of surrounding and decoding target blocks or information of symbols / bins decoded in the previous step, predict the probability of occurrence of the bin according to the determined context model, and perform arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element.At this time, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the context model of the next symbol / bin after determining the context model. Among the information decoded in the entropy decoding unit (310), information regarding prediction is provided to the prediction unit (inter prediction unit (332) and intra prediction unit (331)), and the residual value for which entropy decoding was performed in the entropy decoding unit (310), i.e., quantized transformation coefficients and related parameter information, can be input to the residual processing unit (320). The residual processing unit (320) can derive residual signals (residual blocks, residual samples, residual sample array). Additionally, among the information decoded in the entropy decoding unit (310), information regarding filtering can be provided to the filtering unit (350). Meanwhile, a receiving unit (not shown) that receives a signal output from an encoding device may be further configured as an internal / external element of the decoding device (300), or the receiving unit may be a component of the entropy decoding unit (310). Meanwhile, the decoding device according to the present document may be called a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoding unit (310), and the sample decoder may include at least one of the inverse quantization unit (321), inverse transform unit (322), adder (340), filtering unit (350), memory (360), inter prediction unit (332), and intra prediction unit (331).

[0065] In the inverse quantization unit (321), the quantized transformation coefficients can be inversely quantized to output transformation coefficients. The inverse quantization unit (321) can rearrange the quantized transformation coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device. The inverse quantization unit (321) can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) and obtain transformation coefficients.

[0066] In the inverse conversion unit (322), the conversion coefficients are inversely converted to obtain a residual signal (residual block, residual sample array).

[0067] The prediction unit performs a prediction for the current block and can generate a predicted block containing prediction samples for the current block. Based on information regarding the prediction output from the entropy decoding unit (310), the prediction unit can determine whether an intra prediction or an inter prediction is applied to the current block and can determine a specific intra / inter prediction mode.

[0068] The prediction unit (320) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit may apply intra prediction or inter prediction for a single block, and may also apply intra prediction and inter prediction simultaneously. This may be called combined inter and intra prediction (CIIP). Additionally, the prediction unit may be based on an intra block copy (IBC) prediction mode or a palette mode for predicting a block. The IBC prediction mode or palette mode may be used for content video / video coding, such as in games, for example, screen content coding (SCC). IBC basically performs prediction within the current picture, but it can be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC may use at least one of the inter prediction techniques described in this document. The palette mode can be viewed as an example of intra coding or intra prediction. When the palette mode is applied, information regarding the palette table and palette index can be included in the above image / video information and signaled.

[0069] The intra prediction unit (331) can predict the current block by referring to samples within the current picture. The referenced samples may be located near the current block or away from it, depending on the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit (331) may determine the prediction mode applied to the current block by using the prediction mode applied to the surrounding blocks.

[0070] The inter prediction unit (332) can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, to reduce the amount of motion information transmitted in the inter prediction mode, motion information can be predicted in blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, neighboring blocks may include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit (332) may construct a motion information candidate list based on the neighboring blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter-prediction can be performed based on various prediction modes, and information regarding the prediction may include information indicating the mode of inter-prediction for the current block.

[0071] The adder (340) can generate a restoration signal (restored picture, restored block, restored sample array) by adding the acquired residual signal to the prediction signal (predicted block, predicted sample array) output from the prediction unit (including the inter prediction unit (332) and / or the intra prediction unit (331)). In cases where there is no residual for the block to be processed, such as when a skip mode is applied, the predicted block can be used as the restoration block.

[0072] The addition unit (340) may be called a restoration unit or a restoration block generation unit. The generated restoration signal may be used for intra-predicting the next block to be processed within the current picture, may be output after filtering as described below, or may be used for inter-predicting the next picture.

[0073] Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture decoding process.

[0074] The filtering unit (350) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (350) can generate a modified restored picture by applying various filtering methods to the restored picture, and can transmit the modified restored picture to memory (360), specifically to the DPB of memory (360). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0075] The (modified) restored picture stored in the DPB of the memory (360) can be used as a reference picture in the inter-prediction unit (332). The memory (360) can store motion information of blocks from which motion information within the current picture has been derived (or decoded) and / or motion information of blocks within the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit (260) to be used as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. The memory (360) can store restoration samples of blocks restored within the current picture and transmit them to the intra-prediction unit (331).

[0076] In this document, the embodiments described in the filtering unit (260), inter prediction unit (221), and intra prediction unit (222) of the encoding device (200) may be applied to the filtering unit (350), inter prediction unit (332), and intra prediction unit (331) of the decoding device (300) in the same or corresponding manner.

[0077] As described above, prediction is performed to increase compression efficiency during video coding. Through this, a predicted block containing predicted samples for the current block, which is the block to be coded, can be generated. Here, the predicted block includes predicted samples in the spatial domain (or pixel domain). The predicted block is derived identically by both the encoding device and the decoding device, and the encoding device can increase video coding efficiency by signaling information regarding the residual between the original block and the predicted block (residual information) to the decoding device, rather than the original sample value of the original block itself. The decoding device derives a residual block containing residual samples based on the residual information, can generate a restored block containing restored samples by combining the residual block and the predicted block, and can generate a restored picture containing the restored blocks.

[0078] The above residual information can be generated through transformation and quantization procedures. For example, an encoding device may derive a residual block between the original block and the predicted block, perform a transformation procedure on residual samples (residual sample array) included in the residual block to derive transformation coefficients, perform a quantization procedure on the transformation coefficients to derive quantized transformation coefficients, and signal the related residual information to a decoding device (via a bitstream). Here, the residual information may include information such as value information, position information, transformation technique, transformation kernel, and quantization parameters of the quantized transformation coefficients. The decoding device may perform an inverse quantization / inverse transformation procedure based on the residual information and derive residual samples (or residual blocks). The decoding device may generate a reconstructed picture based on the predicted block and the residual block. The encoding device can also derive a residual block by inversely quantizing / inversely transforming the quantized transform coefficients for reference to inter-predicting of the picture, and generate a restored picture based thereon.

[0079] Intra prediction may represent a prediction that generates prediction samples for the current block based on reference samples within the picture to which the current block belongs (hereinafter, the current picture). When intra prediction is applied to the current block, surrounding reference samples to be used for the intra prediction of the current block may be derived. The surrounding reference samples of the current block may include a sample adjacent to the left boundary of the current block of size nWxnH and a total of 2xnH samples adjacent to the bottom-left, a sample adjacent to the top boundary of the current block and a total of 2xnW samples adjacent to the top-right, and one sample adjacent to the top-left of the current block. Alternatively, the surrounding reference samples of the current block may include multiple columns of upper surrounding samples and multiple rows of left surrounding samples. Additionally, the surrounding reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block of size nWxnH, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the bottom-right of the current block.

[0080] However, some of the surrounding reference samples of the current block may not yet be decoded or may not be available. In this case, the decoder may construct the surrounding reference samples to be used for prediction by substituting the unavailable samples with available samples. Alternatively, the surrounding reference samples to be used for prediction may be constructed through the interpolation of available samples.

[0081] When neighboring reference samples are derived, (i) a prediction sample may be derived based on the average or interpolation of the neighboring reference samples of the current block, and (ii) the prediction sample may be derived based on a reference sample existing in a specific (prediction) direction with respect to the prediction sample among the neighboring reference samples of the current block. Case (i) may be called a non-directional mode or non-angular mode, and case (ii) may be called a directional mode or angular mode.

[0082] In addition, the prediction sample may be generated through interpolation between a first surrounding sample located in the prediction direction of the intra prediction mode of the current block and a second surrounding sample located in the opposite direction of the prediction direction, based on the prediction sample of the current block among the surrounding reference samples. The above-described case may be called Linear Interpolation Intra Prediction (LIP). Additionally, chroma prediction samples may be generated based on luminance samples using a linear model. This case may be called LM mode.

[0083] In addition, a provisional prediction sample of the current block may be derived based on filtered surrounding reference samples, and a prediction sample of the current block may be derived by performing a weighted sum of the provisional prediction sample and at least one reference sample derived according to the intra prediction mode among the existing surrounding reference samples, i.e., unfiltered surrounding reference samples. The above case may be called PDPC (Position dependent intra prediction).

[0084] In addition, intra-prediction coding can be performed by selecting the reference sample line with the highest prediction accuracy among the surrounding multiple reference sample lines of the current block, deriving a prediction sample using a reference sample located in the prediction direction from that line, and signaling the used reference sample line to a decoding device. The above-described case may be referred to as multi-reference line intra-prediction or MRL-based intra-prediction.

[0085] In addition, the current block can be divided into vertical or horizontal subpartitions to perform intra prediction based on the same intra prediction mode, while utilizing surrounding reference samples derived at the subpartition level. That is, in this case, the intra prediction mode for the current block is applied identically to the subpartitions, but by deriving and utilizing surrounding reference samples at the subpartition level, intra prediction performance can be improved depending on the circumstances. This prediction method can be referred to as ISP (intra sub-partitions) based intra prediction.

[0086] The intra prediction methods described above may be referred to as intra prediction types to distinguish them from intra prediction modes. The aforementioned intra prediction types may be referred to by various terms, such as intra prediction techniques or additional intra prediction modes. For example, the aforementioned intra prediction type (or additional intra prediction mode, etc.) may include at least one of the aforementioned LIP, PDPC, MRL, and ISP. General intra prediction methods excluding specific intra prediction types such as LIP, PDPC, MRL, and ISP may be referred to as normal intra prediction types. Normal intra prediction types can be generally applied when specific intra prediction types such as those described above are not applied, and predictions can be performed based on the aforementioned intra prediction modes. Meanwhile, post-processing filtering may be performed on the derived prediction samples as necessary.

[0087] Specifically, the intra-prediction procedure may include an intra-prediction mode / type determination step, a peripheral reference sample derivation step, and an intra-prediction mode / type-based prediction sample derivation step. Additionally, a post-filtering step for the derived prediction samples may be performed as needed.

[0088] When intra prediction is applied, the intra prediction mode applied to the current block can be determined using the intra prediction mode of the surrounding blocks. For example, the decoding device may select one of the MPM candidates within the list of most probable modes (MPM) derived based on the intra prediction mode of the surrounding blocks of the current block (e.g., left and / or upper surrounding blocks) and additional candidate modes based on the received MPM index, or select one of the remaining intra prediction modes not included in the said MPM candidates (and planner modes) based on remaining intra prediction mode information. The said MPM list may be configured to include or not include the planner mode as a candidate. For example, if the said MPM list includes the planner mode as a candidate, the said MPM list may have 6 candidates, and if the said MPM list does not include the planner mode as a candidate, the said MPM list may have 5 candidates. If the above MPM list does not include planar mode as a candidate, a not planar flag (e.g., intra_luma_not_planar_flag) indicating whether the intra prediction mode of the current block is not planar mode may be signaled. For example, the MPM flag may be signaled first, and the MPM index and the not planar flag may be signaled when the value of the MPM flag is 1. Additionally, the MPM index may be signaled when the value of the not planar flag is 1. Here, the reason the above MPM list is configured not to include planar mode as a candidate is not that the planar mode is not an MPM, but rather that since planar mode is always considered as an MPM, the not planar flag is signaled first to check whether it is a planar mode.

[0089] For example, whether the intra prediction mode applied to the current block is among the MPM candidates (and planner mode) or is in remaining mode can be indicated based on an MPM flag (e.g., intra_luma_mpm_flag). A value of 1 for the MPM flag may indicate that the intra prediction mode for the current block is among the MPM candidates (and planner mode), and a value of 0 for the MPM flag may indicate that the intra prediction mode for the current block is not among the MPM candidates (and planner mode). A value of 0 for the not planner flag (e.g., intra_luma_not_planar_flag) may indicate that the intra prediction mode for the current block is planner mode, and a value of 1 for the not planner flag may indicate that the intra prediction mode for the current block is not planner mode. The above MPM index may be signaled in the form of the mpm_idx or intra_luma_mpm_idx syntax element, and the above remaining intra prediction mode information may be signaled in the form of the rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax element. For example, the above remaining intra prediction mode information may point to one of the remaining intra prediction modes among all intra prediction modes that are not included in the above MPM candidates (and planar modes) by indexing them in order of prediction mode number. The above intra prediction mode may be an intra prediction mode for the lumina component (sample). Hereinafter, the intra prediction mode information is the above MPM flag (e.g., intra_luma_mpm_flag), the above not planar flag (e.g., intra_luma_not_planar_flag), the above MPM index (e.g.It may include at least one of mpm_idx or intra_luma_mpm_idx) and the remaining intra-prediction mode information (rem_intra_luma_pred_mode or intra_luma_mpm_remainder). In this document, the MPM list may be referred to by various terms such as MPM candidate list, candModeList, etc. When a MIP is applied to the current block, a separate mpm flag for the MIP (e.g., intra_mip_mpm_flag), an mpm index (e.g., intra_mip_mpm_idx), and remaining intra-prediction mode information (e.g., intra_mip_mpm_remainder) may be signaled, and the not-planar flag is not signaled.

[0090] In other words, generally, when an image is partitioned into blocks, the current block to be coded and its neighboring blocks will have similar image characteristics. Therefore, the current block and neighboring blocks are highly likely to have identical or similar intra prediction modes. Consequently, the encoder can utilize the intra prediction mode of the neighboring blocks to encode the intra prediction mode of the current block.

[0091] For example, an encoder / decoder can construct a list of most probable modes (MPM) for the current block. The above MPM list may also be referred to as an MPM candidate list. Here, MPM may refer to a mode used to improve coding efficiency by considering the similarity between the current block and surrounding blocks during intra-predictive mode coding. As described above, the MPM list may be constructed to include planner modes, or it may be constructed to exclude planner modes. For example, if the MPM list includes planner modes, the number of candidates in the MPM list may be 6. And, if the MPM list does not include planner modes, the number of candidates in the MPM list may be 5.

[0092] The encoder / decoder can configure an MPM list containing 5 or 6 MPMs.

[0093] To construct the MPM list, three types of modes can be considered: default intra modes, neighbor intra modes, and derived intra modes.

[0094] For the above surrounding intra-modes, two surrounding blocks, namely the left surrounding block and the upper surrounding block, can be considered.

[0095] As described above, if the MPM list is configured not to include a planner mode, the planner mode is excluded from the list, and the number of MPM list candidates can be set to 5.

[0096] Additionally, among the intra prediction modes, the non-directional mode (or non-angular mode) may include a DC mode based on the average of neighboring reference samples of the current block or a planar mode based on interpolation.

[0097] When inter prediction is applied, the prediction unit of the encoding device / decoding device can perform inter prediction on a block-by-block basis to derive prediction samples. Inter prediction can be a prediction derived in a manner that is dependent on data elements (e.g., sample values, or motion information) of picture(s) other than the current picture. When inter prediction is applied to the current block, a predicted block (prediction sample array) for the current block can be derived based on a reference block (reference sample array) specified by a motion vector on the reference picture pointed to by the reference picture index. At this time, to reduce the amount of motion information transmitted in the inter prediction mode, the motion information of the current block can be predicted on a block, sub-block, or sample basis, based on the correlation of motion information between surrounding blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction type information (L0 prediction, L1 prediction, Bi prediction, etc.). When inter-prediction is applied, neighboring blocks may include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. The reference picture containing the reference blocks and the reference picture containing the temporal neighboring blocks may be the same or different.The above temporal surrounding blocks may be referred to by names such as collocated reference block or collocated CU (colCU), and the reference picture containing the above temporal surrounding blocks may be referred to as a collocated picture (colPic). For example, a list of motion information candidates may be constructed based on the surrounding blocks of the current block, and a flag or index information indicating which candidate is selected (used) to derive the motion vector and / or reference picture index of the current block may be signaled. Inter-prediction may be performed based on various prediction modes; for example, in the case of skip mode and merge mode, the motion information of the current block may be the same as the motion information of the selected surrounding blocks. In the case of skip mode, unlike merge mode, a residual signal may not be transmitted. In the case of motion vector prediction (MVP) mode, the motion vector of the selected surrounding block is used as a motion vector predictor, and the motion vector difference may be signaled. In this case, the motion vector of the current block can be derived by using the sum of the motion vector predictor and the motion vector difference.

[0098] The above motion information may include L0 motion information and / or L1 motion information depending on the inter-prediction type (L0 prediction, L1 prediction, Bi prediction, etc.). A motion vector in the L0 direction may be called an L0 motion vector or MVL0, and a motion vector in the L1 direction may be called an L1 motion vector or MVL1. A prediction based on an L0 motion vector may be called an L0 prediction, a prediction based on an L1 motion vector may be called an L1 prediction, and a prediction based on both the L0 motion vector and the L1 motion vector may be called a pair (Bi) prediction. Here, the L0 motion vector may represent a motion vector associated with reference picture list L0 (L0), and the L1 motion vector may represent a motion vector associated with reference picture list L1 (L1). Reference picture list L0 may include pictures that are prior to the current picture in output order as reference pictures, and reference picture list L1 may include pictures that are subsequent to the current picture in output order. The aforementioned previous pictures may be referred to as forward (reference) pictures, and the aforementioned subsequent pictures may be referred to as reverse (reference) pictures. The reference picture list L0 may include additional pictures that are output later than the current picture as reference pictures. In this case, within the reference picture list L0, the previous pictures may be indexed first, and the subsequent pictures may be indexed next. The reference picture list L1 may include additional pictures that are output earlier than the current picture as reference pictures. In this case, within the reference picture list L1, the subsequent pictures may be indexed first, and the previous pictures may be indexed next. Here, the output order may correspond to the POC (picture order count) order.

[0099] Figure 4 illustrates an exemplary hierarchical structure for a coded image / video.

[0100] Referring to Figure 4, the coded image / video is divided into a VCL (video coding layer) that handles the decoding processing of the image / video and the image / video itself, a subsystem that transmits and stores the encoded information, and a NAL (network abstraction layer) that exists between the VCL and the subsystem and is responsible for network adaptation functions.

[0101] In VCL, VCL data containing compressed image data (slice data) can be generated, or parameter sets containing information such as Picture Parameter Set (PPS), Sequence Parameter Set (SPS), and Video Parameter Set (VPS), or SEI (Supplemental Enhancement Information) messages that are additionally required in the decoding process of the image can be generated.

[0102] In NAL, a NAL unit can be created by adding header information (NAL unit header) to the Raw Byte Sequence Payload (RBSP) generated in VCL. In this case, the RBSP refers to slice data, parameter sets, SEI messages, etc. generated in VCL. The NAL unit header may include NAL unit type information specified according to the RBSP data included in the NAL unit.

[0103] As illustrated in FIG. 4, NAL units can be classified into VCL NAL units and Non-VCL NAL units depending on the RBSP generated in VCL. A VCL NAL unit may refer to a NAL unit containing information about an image (slice data), and a Non-VCL NAL unit may refer to a NAL unit containing information necessary to decode an image (parameter set or SEI message).

[0104] The aforementioned VCL NAL unit and Non-VCL NAL unit can be transmitted over a network by attaching header information according to the data specifications of the underlying system. For example, the NAL unit can be transformed into a data format of a specified specification, such as H.266 / VVC file format, RTP (Real-time Transport Protocol), TS (Transport Stream), etc., and transmitted over various networks.

[0105] As described above, the NAL unit type can be determined according to the RBSP data structure included in the NAL unit, and information about this NAL unit type can be stored in the NAL unit header and signaled.

[0106] For example, NAL units can be broadly classified into VCL NAL unit types and Non-VCL NAL unit types depending on whether they contain information about the image (slice data). VCL NAL unit types can be classified according to the properties and types of the picture included in the VCL NAL unit, while Non-VCL NAL unit types can be classified according to the types of parameter sets.

[0107] The following is an example of a NAL unit type specified according to the type of parameter set included in the Non-VCL NAL unit type.

[0108] - APS (Adaptation Parameter Set) NAL unit: Type for the NAL unit containing the APS

[0109] - DPS (Decoding Parameter Set) NAL unit: Type for the NAL unit containing the DPS

[0110] - VPS (Video Parameter Set) NAL unit: Type for the NAL unit containing the VPS

[0111] - SPS (Sequence Parameter Set) NAL unit: Type for the NAL unit containing the SPS

[0112] - PPS(Picture Parameter Set) NAL unit: Type for the NAL unit containing the PPS

[0113] - PH (Picture header) NAL unit: Type for NAL unit containing PH

[0114] The above-described NAL unit types have syntax information for the NAL unit type, and said syntax information can be stored in the NAL unit header and signaled. For example, said syntax information may be nal_unit_type, and NAL unit types may be specified by the nal_unit_type value.

[0115] Meanwhile, as described above, a single picture may include multiple slices, and a single slice may include a slice header and slice data. In this case, a picture header may be additionally added for multiple slices (slice header and slice data set) within a single picture. The picture header (picture header syntax) may include information / parameters that can be commonly applied to the picture. In this document, a tile group may be used interchangeably or replaced with a slice. Additionally, in this document, a tile group header may be used interchangeably or replaced with a slice header.

[0116] The slice header (slice header syntax) may include information / parameters that can be commonly applied to the slice. The APS (APS syntax) or PPS (PPS syntax) may include information / parameters that can be commonly applied to one or more slices or pictures. The SPS (SPS syntax) may include information / parameters that can be commonly applied to one or more sequences. The VPS (VPS syntax) may include information / parameters that can be commonly applied to multiple layers. The DPS (DPS syntax) may include information / parameters that can be commonly applied to the entire video. The DPS may include information / parameters related to the concatenation of the CVS (coded video sequence). In this document, the term High level syntax (HLS) may include at least one of the above APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, picture header syntax, and slice header syntax.

[0117] In this document, the video information encoded from an encoding device to a decoding device and signaled in the form of a bitstream includes not only information related to picture partitioning, intra / inter prediction information, residual information, and in-loop filtering information, but may also include information included in the slice header, information included in the picture header, information included in the APS, information included in the PPS, information included in the SPS, information included in the VPS, and / or information included in the DPS. Additionally, the video information may further include information from the NAL unit header.

[0118] Meanwhile, to compensate for the difference between the original image and the restored image caused by errors occurring during compression encoding processes such as quantization, an in-loop filtering procedure may be performed on the restored samples or restored pictures as described above. As described above, in-loop filtering may be performed in the filter section of the encoding device and the filter section of the decoding device, and a deblocking filter, SAO, and / or an adaptive loop filter (ALF) may be applied. For example, the ALF procedure may be performed after the deblocking filtering procedure and / or SAO procedure are completed. However, even in this case, the deblocking filtering procedure and / or SAO procedure may be omitted.

[0119] FIG. 5 is a flowchart schematically illustrating an example of an ALF procedure. The ALF procedure disclosed in FIG. 5 may be performed in an encoding device and a decoding device. In this document, the coding device may include the encoding device and / or the decoding device.

[0120] Referring to FIG. 5, the coding device derives a filter for ALF (S500). The filter may include filter coefficients. The coding device may determine whether to apply ALF, and if it determines to apply ALF, it may derive a filter including the filter coefficients for ALF. The filter (coefficients) for ALF or the information for deriving the filter (coefficients) for ALF may be referred to as ALF parameters. Information regarding whether to apply ALF (e.g., an ALF availability flag) and ALF data for deriving the filter may be signaled from the encoding device to the decoding device. The ALF data may include information for deriving the filter for ALF. Additionally, as an example, for hierarchical control of ALF, the ALF availability flag may be signaled at the SPS, picture header, slice header, and / or CTB levels, respectively.

[0121] To derive a filter for the ALF, the activity and / or directivity of the current block (or ALF target block) is derived, and the filter may be derived based on the activity and / or directivity. For example, the ALF procedure may be applied in units of 4x4 blocks (based on the luma component). The current block or ALF target block may be, for example, a CU, or a 4x4 block within a CU. Specifically, for example, filters for the ALF may be derived based on first filters derived from information included in the ALF data and predefined second filters, and the coding device may select one of the filters based on the activity and / or directivity. The coding device may use the filter coefficients included in the selected filter for the ALF.

[0122] The coding device performs filtering based on the filter (S510). Modified reconstructed samples can be derived based on the filtering. For example, the filter coefficients within the filter may be arranged or assigned according to the filter shape, and the filtering may be performed on the reconstructed samples within the current block. Here, the reconstructed samples within the current block may be reconstructed samples after the deblocking filter procedure and the SAO procedure are completed. For example, one filter shape may be used, or one filter shape may be selected and used from a predetermined plurality of filter shapes. For example, the filter shape applied to the luminance component and the filter shape applied to the chroma component may be different. For example, a 7x7 diamond filter shape may be used for the luminance component, and a 5x5 diamond filter shape may be used for the chroma component.

[0123] Figures 6a and 6b show examples of ALF filter shapes.

[0124] FIG. 6a shows a 7x7 diamond filter shape, and FIG. 6b shows a 5x5 diamond filter shape. In FIG. 6a and FIG. 6b, Cn within the filter shape represents a filter coefficient. If n is the same in Cn, it indicates that the same filter coefficient can be assigned. In this document, the location and / or unit where filter coefficients are assigned according to the filter shape of the ALF may be called a filter tab. In this case, one filter coefficient may be assigned to each filter tab, and the arrangement of the filter tabs may correspond to the filter shape. A filter tab located at the center of the filter shape may be called a center filter tab. Two filter tabs with the same n value located at mutually corresponding positions relative to the center filter tab may be assigned the same filter coefficient. For example, in the case of a 7x7 diamond filter shape, it includes 25 filter tabs, and since filter coefficients C0 to C11 are assigned in a centrally symmetrical form, filter coefficients can be assigned to the 25 filter tabs using only 13 filter coefficients. Additionally, for example, in the case of a 5x5 diamond filter shape, it includes 13 filter tabs, and since filter coefficients C0 through C5 are assigned in a centrally symmetrical form, filter coefficients can be assigned to the 13 filter tabs using only 7 filter coefficients. For example, to reduce the amount of data regarding information on the signaled filter coefficients, 12 of the 13 filter coefficients for a 7x7 diamond filter shape may be (explicitly) signaled, and 1 filter coefficient may be (implicitly) derived. Also, for example, 6 of the 7 filter coefficients for a 5x5 diamond filter shape may be (explicitly) signaled, and 1 filter coefficient may be (implicitly) derived.

[0125] According to one embodiment of the present document, ALF parameters used for the ALF procedure may be signaled through an adaptation parameter set (APS). The ALF parameters may be derived from filter information for the ALF or from ALF data.

[0126] As mentioned above, ALF is a type of in-loop filtering technique that can be applied in video / image coding. ALF can be performed using Wiener-based adaptive filters. This may be intended to minimize the mean square error (MSE) between the original samples and the decoded samples (or reconstructed samples). The high-level design for the ALF tool may incorporate syntax elements accessible from the SPS and / or slice headers (or tile group headers).

[0127] Figure 7 shows an example of the hierarchical structure of ALF data.

[0128] Referring to FIG. 7, a coded video sequence (CVS) may include an SPS, one or more PPSs, and one or more subsequent coded pictures. Each coded picture may be divided into rectangular regions. These rectangular regions may be referred to as tiles. One or more tiles may be combined to form a tile group or a slice. In this case, a tile group header may be linked to a PPS, and the PPS may be linked to an SPS. According to the conventional method, the ALF data (ALF parameters) was included in the tile group header. Given that a video consists of multiple pictures and a picture contains multiple tiles, the frequent signaling of ALF data (ALF parameters) at the tile group level had a problem of reducing coding efficiency.

[0129] According to one embodiment proposed in this document, the ALF parameter can be included in the APS and signaled as follows.

[0130] Figure 8 shows another example of the hierarchical structure of ALF data.

[0131] Referring to FIG. 8, an APS is defined, and the APS can carry necessary ALF data (ALF parameters). Furthermore, the APS may have a self-identification parameter and ALF data. The self-identification parameter of the APS may include an APS ID. That is, in addition to the ALF data field, the APS may include information indicating the APS ID. A tile group header or a slice header may reference the APS using APS index information. In other words, the tile group header or the slice header may include APS index information and can perform an ALF procedure for a target block based on the ALF data (ALF parameters) contained in the APS having the APS ID pointed to by the APS index information. Here, the APS index information may also be referred to as APS ID information.

[0132] Additionally, the above SPS may include a flag that allows the use of ALF. For example, when CVS begins, the SPS is checked, and the flag within the SPS may be checked. For example, the SPS may include the syntax of Table 1 below. The syntax of Table 1 may be part of the SPS.

[0133]

[0134] The semantics of the syntax elements included in the syntax of Table 1 above can be represented, for example, as shown in the following table.

[0135]

[0136] That is, the sps_alf_enabled_flag syntax element can indicate whether ALF is available based on whether its value is 0 or 1. The sps_alf_enabled_flag syntax element can be called the ALF availability flag (which may be called the first ALF availability flag) and can be included in the SPS. That is, the ALF availability flag can be signaled at the SPS (or SPS level). If the value of the ALF availability flag signaled at the SPS is 1, it can be determined that ALF is available by default for pictures within the CVS referencing the SPS. Meanwhile, as described above, ALF can be individually turned on / off by signaling an additional availability flag at a level lower than the SPS.

[0137] For example, if the ALF tool is available for CVS, an additional availability flag (which may be called a second ALF availability flag) may be signaled in the tile group header or slice header. The second ALF availability flag may be parsed / signaled, for example, when ALF is available at the SPS level. If the value of the second ALF availability flag is 1, the ALF data may be parsed through the tile group header or the slice header. For example, the second ALF availability flag may specify an ALF availability condition regarding luminance and chroma components. The ALF data may be accessed through APS ID information.

[0138]

[0139]

[0140] The semantics of the syntax elements included in the syntax of Table 3 or Table 4 above can be represented, for example, as shown in the following tables.

[0141]

[0142]

[0143] The second ALF enabled flag may include the tile_group_alf_enabled_flag syntax element or the slice_alf_enabled_flag syntax element.

[0144] Based on the above APS ID information (e.g., tile_group_aps_id syntax element or slice_aps_id syntax element), the APS referenced by the corresponding tile group or corresponding slice can be identified. The above APS may include ALF data.

[0145] Meanwhile, the structure of an APS containing ALF data can be described based on, for example, the following syntax and semantics. The syntax in Table 7 may be part of an APS.

[0146]

[0147]

[0148] As described above, the adaptation_parameter_set_id syntax element may represent the identifier of the corresponding APS. That is, the APS can be identified based on the adaptation_parameter_set_id syntax element. The adaptation_parameter_set_id syntax element may be referred to as APS ID information. Additionally, the APS may include an ALF data field. The ALF data field may be parsed / signaled after the adaptation_parameter_set_id syntax element.

[0149] Additionally, for example, APS extension flags (e.g., aps_extension_flag syntax elements) may be parsed / signaled in the APS. The said APS extension flags may indicate whether APS extension data flags (aps_extension_data_flag) syntax elements exist. The said APS extension flags may be used, for example, to provide extension points for later versions of the VVC standard.

[0150] Key processing / handling of ALF information can be performed in a slice header or a tile group header. The ALF data field described above may include information regarding the processing of the ALF filter. For example, information that can be extracted from the ALF data field may include information on the number of filters used, information indicating whether the ALF is applied only to the luminance component, information regarding the color component, information regarding exponential golomb (EG) parameters and / or delta values ​​of filter coefficients, etc.

[0151] Meanwhile, an ALF data field may include ALF data syntax as follows, for example.

[0152]

[0153] The semantics of the syntax elements included in the syntax of Table 9 above can be represented, for example, as shown in the following table.

[0154]

[0155]

[0156]

[0157]

[0158] For example, parsing ALF data through a tile group header or slice header can begin by first parsing / signaling the alf_chroma_idc syntax element. The alf_chroma_idc syntax element can have values ​​ranging from 0 to 3. These values ​​may indicate whether the ALF-based filtering procedure is applied only to the luminance component or to a combination of luminance and chroma components. Once the availability (availability parameters) for each component is determined, information regarding the number of luminance (component) filters can be parsed. For example, the maximum number of filters that can be used can be set to 25. If the number of luminance filters being signaled is at least one, index information for each filter in the range from 0 to the maximum number of filters (e.g., 25, which may alternatively be known as class) can be parsed / signaled. This implies that each class (i.e., from 0 to the maximum number of filters) is associated with a filter index. When the filter to be used for each class is labeled based on the above filter index, a flag (e.g., alf_luma_coeff_delta_flag) can be parsed / signaled. The above flag can be used to determine whether flag information regarding the prediction of ALF luma filter coefficient delta values ​​(e.g., alf_luma_coeff_delta_prediction_flag) exists in the slice header or tile group header.

[0159] If the number of luminance filters signaled by the alf_luma_num_filters_signalled_minus1 syntax element is greater than 0 and the value of the alf_luma_coeff_delta_flag syntax element is 0, this may mean that the alf_luma_coeff_delta_prediction_flag syntax element exists in the slice header or tile group header and its status can be evaluated. If the status of the alf_luma_coeff_delta_prediction_flag syntax element is 1, this may mean that the luminance filter coefficients are predicted from the previous luminance (filter) coefficients. If the status of the alf_luma_coeff_delta_prediction_flag syntax element is 0, this may mean that the luminance filter coefficients are not predicted from the deltas of the previous luminance (filter) coefficients.

[0160] If delta filter coefficients (e.g., alf_luma_coeff_delta_abs) are coded based on exponential Gollum code, the order k of the exponential Gollum (EG) code may need to be determined in order to decode the delta-luma filter coefficients (e.g., alf_luma_coeff_delta_abs). This information may be necessary to decode the filter coefficients. The order of the exponential Gollum code may be expressed as EG(k). To determine EG(k), the alf_luma_min_eg_order_minus1 syntax element may be parsed / signaled. The alf_luma_min_eg_order_minus1 syntax element may be an entropy-coded syntax element. The alf_luma_min_eg_order_minus1 syntax element may represent the smallest order of the EG used for decoding the delta-luma filter coefficients. For example, the value of the alf_luma_min_eg_order_minus1 syntax element may be a value within the range from 0 to 6. After the alf_luma_min_eg_order_minus1 syntax element is parsed / signaled, the alf_luma_eg_order_increase_flag syntax element may be parsed / signaled. If the value of the alf_luma_eg_order_increase_flag syntax element is 1, it indicates that the order of the EG represented by the alf_luma_min_eg_order_minus1 syntax element increases by 1. If the value of the alf_luma_eg_order_increase_flag syntax element is 0, it indicates that the order of the EG represented by the alf_luma_min_eg_order_minus1 syntax element does not increase. The order of the EG may be represented by the index of the EG.The EG order (or EG index) (regarding the luma component) based on the above alf_luma_min_eg_order_minus1 syntax element and the above alf_luma_eg_order_increase_flag syntax element can be determined, for example, as follows.

[0161]

[0162] Based on the above determination procedure, expGoOrderY can be derived as expGoOrderY=KminTab. Through this, an array including EG orders can be derived, which can be used by a decoding device. The expGoOrderY can represent the EG order (or EG index).

[0163] There may be a pre-defined Gollum order index (i.e., golombOrderIdxY). The pre-defined Gollum order can be used to determine the final Gollum order for coding the coefficients.

[0164] For example, the above-mentioned predefined Gollum order can be constructed as, for example, as in the following mathematical formula.

[0165]

[0166] Here, the order k = expGoOrderY[golombOrderIdxY[j]], where j represents the j-th signaling filter coefficient. For example, if j=2, i.e., the 3rd filter coefficient, golomborderIdxY[2] = 1, and therefore k = expGoOrderY[1].

[0167] In this case, for example, if the value of the alf_luma_coeff_delta_flag syntax element is true, i.e., 1, the alf_luma_coeff_flag syntax element may be signaled for every signaling filter. The alf_luma_coeff_flag syntax element indicates whether the lumina filter coefficients are (explicitly) signaled.

[0168] When the state of the above-mentioned EG order and the aforementioned related flags (e.g., alf_luma_coeff_delta_flag, alf_luma_coeff_flag, etc.) is determined, the difference information and sign information of the luminance filter coefficients can be parsed / signaled (i.e., when alf_luma_coeff_flag indicates true). The absolute delta value information (alf_luma_coeff_delata_abs syntax element) for each of the 12 filter coefficients can be parsed / signaled. Furthermore, if the alf_luma_coeff_delata_abs syntax element has a value, the sign information (alf_luma_coeff_delta_sign syntax element) can be parsed / signaled. The information including the difference information and sign information of the above-mentioned luminance filter coefficients can be referred to as information regarding the above-mentioned luminance filter coefficients.

[0169] The deltas of the above filter coefficients can be determined and stored along with the sign. In this case, the deltas of the above signed filter coefficients can be stored in an array form and can be represented as filterCoefficients. The deltas of the above filter coefficients can be called delta-luminance coefficients, and the deltas of the above signed filter coefficients can be called signed delta-luminance coefficients.

[0170] To determine the final filter coefficients from the delta-luminance coefficients having the above sign, the (luminance) filter coefficients can be updated as follows:

[0171]

[0172] Here, j can represent a filter coefficient index, and sigFiltIdx can represent a signaling filter index. j={0,...,11} and sigFiltIdx = {0, ,alf_luma_filters_signaled_minus1}.

[0173] The above coefficients are the final AlfCoeff L It can be copied to [filtIdx][j]. Here, filtidx = 0,...,24 and j = 0,...,11.

[0174] The delta-luminosity coefficients with the above sign for a given filter index can be used to determine the first 12 filter coefficients. The 13th filter coefficient of a 7x7 filter can be determined, for example, based on the following mathematical formula. The 13th filter coefficient may represent the filter coefficient of the center tap described above.

[0175]

[0176] Here, the filter coefficient index 12 can represent the 13th filter coefficient. For reference, since the filter coefficient index starts from 0, the value 12 can represent the 13th filter coefficient.

[0177] For example, to ensure bitstream conformance, the final filter coefficients AlfCoeff L The range of [filtIdx][k] values ​​is -2 when k is 0, ..., 11. 7 From 2 7 It can be up to -1, and if k is 12, it ranges from 0 to 2 8It can be up to -1. Here, the above k can be replaced with j.

[0178] When processing is performed on the luminance component, processing on the chroma component may be performed based on the alf_chroma_idc syntax element. If the value of the alf_chroma_idc syntax element is greater than 0, the minimum EG order information for the chroma component (e.g., the alf_chroma_min_eg_order_minus1 syntax element) may be parsed / signaled. According to the embodiment of this document described above, a 5x5 diamond filter shape may be used for the chroma component, so in this case, the maximum Gollum index may be 2. In this case, the EG order (or EG index) for the chroma component may be determined as, for example, as follows.

[0179]

[0180] Based on the above judgment procedure, expGoOrderC can be derived as expGoOrderC=KminTab. Through this, an array including EG orders can be derived, which can be used by a decoding device. The EG order (or EG index) regarding the chroma component of expGoOrderC can be represented.

[0181] There may be a pre-defined gollum order index (golombOrderIdxC). The pre-defined gollum order can be used to determine the final golomb order for coding the coefficients.

[0182] For example, the above-mentioned predefined Gollum order can be constructed as, for example, as in the following mathematical formula.

[0183]

[0184] Here, the order k = expGoOrderC[golombOrderIdxC[j]], where j represents the j-th signaling filter coefficient. For example, if j=2, i.e., the 3rd filter coefficient, then golomborderIdxY[2] = 1, and thus k = expGoOrderC[1].

[0185] Based on this, absolute value information and sign information of the chroma filter coefficients can be parsed / signaled. Information including the absolute value information and sign information of the chroma filter coefficients can be referred to as information regarding the chroma filter coefficients. For example, a 5x5 diamond filter shape may be applied to a chroma component, in which case delta absolute value information (alf_chroma_coeff_abs syntax element) for each of the six (chroma component) filter coefficients can be parsed / signaled. Furthermore, if the value of the alf_chroma_coeff_abs syntax element is greater than 0, sign information (alf_chroma_coeff_sign syntax element) can be parsed / signaled. For example, the six chroma filter coefficients can be derived based on the information regarding the chroma filter coefficients. In this case, the seventh chroma filter coefficient can be determined, for example, based on the following mathematical formula. The above 7th filter coefficient may represent the filter coefficient of the center tab described above.

[0186]

[0187] Here, the filter coefficient index 6 can represent the 7th filter coefficient. For reference, since the filter coefficient index starts from 0, the value 6 can represent the 7th filter coefficient.

[0188] For example, to ensure bitstream conformance, the final filter coefficients AlfCoeff CThe range of [filtIdx][k] values ​​is -2 when k is 0, ◎, or 5. 7 From 2 7 It can be up to -1, and if k is 6, it ranges from 0 to 2 8 It can be up to -1. Here, the above k can be replaced with j.

[0189] Once (luminance / chroma) filter coefficients are derived, ALF-based filtering can be performed based on said filter coefficients or a filter containing said filter coefficients. As previously described, modified reconstructed samples can be derived through this process. Additionally, multiple filters may be derived, and the filter coefficients of one of said filters may be used for the ALF procedure. For example, one of said filters may be indicated based on signaled filter selection information. Alternatively, for example, one of said filters may be selected based on the activity and / or directionality of the current block or the ALF target block, and the filter coefficients of said selected filter may be used for the ALF procedure.

[0190] Meanwhile, as described above, LMCS (luma mapping with chroma scaling) may be applied to increase coding efficiency. LMCS may be referred to as loop reshaper (reshaping). To increase coding efficiency, the control of LMCS and / or the signaling of LMCS-related information may be performed hierarchically.

[0191] FIG. 9 illustrates an exemplary hierarchical structure of a CVS according to one embodiment of the present document. A coded video sequence (CVS) may include a sequence parameter set (SPS), a picture parameter set (PPS), a tile group header, tile data, and / or CTU(s). Here, the tile group header and tile data may be referred to as the slice header and slice data, respectively.

[0192] The SPS may natively contain flags to enable tools for use in CVS. Additionally, the SPS may be referenced by a PPS containing information about parameters that change per picture. Each encoded picture may contain one or more encoded rectangular domain tiles. These tiles may be grouped into raster scans that form tile groups. Each tile group is encapsulated by header information called a tile group header. Each tile consists of a CTU containing encoded data. Here, the data may include original sample values, predicted sample values, and their luminance and chroma components (luminance predicted sample values ​​and chroma predicted sample values).

[0193] FIG. 10 illustrates an exemplary LMCS structure according to one embodiment of the present document. The LMCS structure (1000) of FIG. 10 may include an in-loop mapping portion (1010) of luminar components based on adaptive piecewise linear (adaptive PWL) models and a luminar-dependent chroma residual scaling portion (1020) for chromar components. The inverse quantization and inverse transform (1011), restoration (1012), and intra prediction (1013) blocks of the in-loop mapping portion (1010) represent processes applied in the mapped (reshaped) domain. The loop filters (1015), motion compensation or inter-prediction (1017) blocks of the in-loop mapping section (1010), and the restoration (1022), intra-prediction (1023), motion compensation or inter-prediction (1024), and loop filters (1025) blocks of the chroma residual scaling section (1020) represent processes applied in the original (non-mapped, unreshaped) domain.

[0194] As described in FIG. 10, when LMCS is enabled, at least one of an inverse reshaping (mapping) process (1014), a forward reshaping (mapping) process (1018), and a chroma scaling process (1021) may be applied. For example, the inverse reshaping process may be applied to (restored) luminance samples (or luminance samples or an array of luminance samples) of the restored picture. The inverse reshaping process may be performed based on the piecewise function (inverse) index of the luminance samples. The piecewise function (inverse) index may identify the piece (or part) to which the luminance samples belong. The output of the inverse reshaping process is a modified (restored) luminance sample (or modified luminance samples or an array of modified luminance samples). LMCS may be enabled or disabled at a tile group (or slice), picture, or a higher level.

[0195] A forward reshaping process and / or a chroma scaling process may be applied to generate a restored picture. The picture may contain luminance samples and chroma samples. A restored picture containing luminance samples may be referred to as a restored luminance picture, and a restored picture containing chroma samples may be referred to as a restored chroma picture. A combination of a restored luminance picture and a restored chroma picture may be referred to as a restored picture. A restored luminance picture may be generated based on a forward reshaping process. For example, if inter-prediction is applied to the current block, forward reshaping is applied to the luminance prediction samples derived based on the (restored) luminance samples of the reference picture. Since the (restored) luminance samples of the reference picture are generated based on an inverse reshaping process, forward reshaping can be applied to the luminance prediction samples to derive a reshaped (mapped) luminance prediction sample. The forward reshaping process can be performed based on the partial function index of the luminance prediction sample. The partial function index can be derived based on the value of the luminance prediction sample of the reference picture used for inter-prediction or the value of the luminance sample. A restoration sample can be generated based on the (reshaped / mapped) luminance prediction sample. An inverse reshaping (mapping) process can be applied to the restoration sample. The restoration sample to which the inverse reshaping (mapping) process has been applied may be called the inverse reshaped (mapped) restoration sample. Additionally, the inverse reshaped (mapped) restoration sample may simply be called the reshaped (mapped) restoration sample. If intra-prediction (or IBC (intra block copy)) is applied to the current block, forward mapping may not be necessary for the prediction sample(s) of the current block because the inverse reshaping process has not yet been applied to the restoration samples of the referenced current picture.In the restored luma picture, (restored) luma samples can be generated based on (reshaped) luma prediction samples and corresponding luma residual samples.

[0196] A restored chroma picture can be generated based on the chroma scaling process. For example, the (restored) chroma samples in the restored chroma picture are the chroma prediction samples and chroma residual samples (c) in the current block. res It can be derived based on ). Chroma residual sample (c res ) is the (scaled) chroma residual sample (c) for the current block resScale It is derived based on ) and the chroma residual scaling factor (cScaleInv can be referred to as varScale). The chroma residual scaling factor can be calculated based on the reshaped luminance predicted sample values ​​in the current block. For example, the scaling factor is the reshaped luminance predicted sample values ​​(Y' pred ) average luma value(ave(Y' pred It can be calculated based on )). For reference, the (scaled) chroma residual sample derived based on inverse transform / inverse quantization in Fig. 10 is c resScale , the chroma residual sample derived by performing an (inverse) scaling procedure on the above (scaled) chroma residual sample is c res It can be referred to as.

[0197] FIG. 11 illustrates an LMCS structure according to another embodiment of the present document. FIG. 11 will be described with reference to FIG. 10. Here, the difference between the LMCS structure (1100) of FIG. 11 and the LMCS structure (1000) of FIG. 10 will be mainly described. The in-loop mapping portion (1110) and the luminance-dependent chroma residual scaling portion (1120) of FIG. 11 may operate identically or similarly to the in-loop mapping portion (1010) and the luminance-dependent chroma residual scaling portion (1020) of FIG. 10.

[0198] Referring to Fig. 11, a chroma residual scaling factor can be derived based on the luminance reconstruction samples. In this case, the average luminance value (avgY) is derived based on the surrounding luminance reconstruction samples outside the reconstruction block, rather than the luminance reconstruction samples inside the reconstruction block. r ) can be obtained and the above average luminance value (avgY r A chroma residual scaling factor can be derived based on ). Here, the surrounding luminance reconstruction samples may be surrounding luminance reconstruction samples of the current block, or surrounding luminance reconstruction samples of a virtual pipeline data unit (VPDU) containing the current block. For example, if intra prediction is applied to the target block, reconstruction samples may be derived based on prediction samples derived from the intra prediction. Also, for example, if inter prediction is applied to the target block, forward mapping may be applied to prediction samples derived from the inter prediction, and reconstruction samples may be generated based on the reshaped (or forward-mapped) luminance prediction samples.

[0199] Video / image information signaled via a bitstream may include LMCS parameters (information regarding LMCS). LMCS parameters may be composed of HLS (high-level syntax, including slice header syntax), etc. A detailed description of the LMCS parameters and their composition will be provided later. As previously stated, the syntax tables described in this document (and the embodiments below) may be configured / encoded at the encoder stage and signaled to the decoder stage via a bitstream. The decoder may parse / decode information regarding LMCS (in the form of syntax components) from the syntax tables. One or more embodiments described below may be combined. The encoder may encode the current picture based on information regarding LMCS, and the decoder may decode the current picture based on information regarding LMCS.

[0200] In-loop mapping of luminance components can adjust the dynamic range of the input signal by redistributing codewords across the dynamic range to improve compression efficiency. For luminance mapping, a forward mapping (reshaping) function (FwdMap) and an inverse mapping (reshaping) function (InvMap) corresponding to the forward mapping function (FwdMap) may be used. The forward mapping function (FwdMap) may be signaled using partial linear models, for example, the partial linear models may have 16 pieces or bins. The pieces may have the same length. In one example, the inverse mapping function (InvMap) may not be signaled separately but may instead be derived from the forward mapping function (FwdMap). That is, the inverse mapping may be a function of the forward mapping. For example, an inverse mapping function can be a function obtained by reflecting the forward mapping function across y=x.

[0201] In-loop (luminance) reshaping can be used to map input luminance values ​​(samples) in the reshaped domain to modified values. The reshaped values ​​can be encoded and, after restoration, mapped back to the original (unmapped, unreshaped) domain. Chroma residual scaling can be applied to compensate for the difference between the luminance signal and the chroma signal. In-loop reshaping can be performed by specifying high-level syntax for the reshaper model. The reshaper model syntax can signal a partial linear model (PWL model). A forward lookup table (FwdLUT) and / or an inverse lookup table (InvLUT) can be derived based on the partial linear model. For example, if a forward lookup table (FwdLUT) is derived, an inverse lookup table (InvLUT) can be derived based on the forward lookup table (FwdLUT). The forward lookup table (FwdLUT) is the input luminance values ​​Y i The changed values ​​Y r Mapped to, and the inverse lookup table (InvLUT) is the restored values ​​Y based on the changed values. r restored values ​​Y' i It can be mapped to. Restored values ​​Y' i are input luma values ​​Y i It can be derived based on.

[0202] In one example, the SPS may include the syntax of Table 13 below. The syntax of Table 16 may include sps_reshaper_enabled_flag as a tool enabling flag. Here, sps_reshaper_enabled_flag can be used to specify whether the reshaper is used in the coded video sequence (CVS). That is, sps_reshaper_enabled_flag may be a flag that enables reshaping in the SPS. In one example, the syntax of Table 16 may be part of the SPS.

[0203]

[0204] In one example, the semantics that sps_seq_parameter_set_id and sps_reshaper_enabled_flag can represent are as shown in Table 17 below.

[0205]

[0206] In one example, a tile group header or a slice header may include the syntax of Table 18 or Table 19 below.

[0207]

[0208]

[0209] The semantics of the syntax elements included in the syntax of Table 18 or Table 19 above may include, for example, the items disclosed in the following tables.

[0210]

[0211]

[0212] As an example, when sps_reshaper_enabled_flag is parsed, the tile group header may parse additional data used to construct lookup tables (FwdLUT and / or InvLUT) (e.g., information contained in Table 18 or 19 above). To do this, the status of the SPS reshaper flags may be checked in the slice header or tile group header. If sps_reshaper_enabled_flag is true (or 1), an additional flag, tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag), may be parsed. The purpose of tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) may be to indicate the presence of a reshaper model. For example, if tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) is true (or 1), it may be indicated that a reshaper exists for the current tile group (or current slice). If tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) is false (or 0), it may be indicated that a reshaper does not exist for the current tile group (or current slice).

[0213] If a reshaper exists and is enabled on the current tile group (or current slice), the reshaper model (e.g., tile_group_reshaper_model() or slice_reshaper_model()) can be processed, and additionally, an additional flag, tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag), can also be parsed. tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) can indicate whether the reshaper model is used on the current tile group (or slice). For example, if tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is 0 (or false), the reshaper model can be indicated as not being used on the current tile group (or current slice). If tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is 1 (or true), the reshaper model can be indicated as being used for the current tile group (or slice).

[0214] As one example, tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) may be true (or 1) and tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) may be false (or 0). This means that a reshaper model exists but is not currently being used in a tile group (or slice). In this case, the reshaper model can be used in the next tile groups (or slices). As another example, tile_group_reshaper_enable_flag may be true (or 1) and tile_group_reshaper_model_present_flag may be false (or 0).

[0215] When the reshaper model (e.g., tile_group_reshaper_model() or slice_reshaper_model()) and tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) are parsed, it can be determined (evaluated) whether conditions necessary for chroma scaling exist. The conditions may include condition 1 (that the current tile group / slice is not intra-coded) and / or condition 2 (that the current tile group / slice is not split into two distinct coding quad tree structures for luminance and chroma, i.e., that the current tile group / slice is not a dual tree structure). If condition 1 and / or condition 2 is true and / or tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is true (or 1), tile_group_reshaper_chroma_residual_scale_flag (or slice_reshaper_chroma_residual_scale_flag) may be parsed. If tile_group_reshaper_chroma_residual_scale_flag (or slice_reshaper_chroma_residual_scale_flag) is enabled (if 1 or true), it may be indicated that chroma residual scaling is enabled for the current tile group (or slice). If tile_group_reshaper_chroma_residual_scale_flag (or slice_reshaper_chroma_residual_scale_flag) is disabled (0 or false), it may indicate that chroma residual scaling is disabled for the current tile group (or slice).

[0216] The purpose of the reshaping described above is to parse the data necessary to construct lookup tables (FwdLUT and / or InvLUT). In one example, the lookup tables constructed based on the parsed data can divide the distribution of acceptable ranges of luminance values ​​into multiple bins (e.g., 16). Thus, luminance values ​​within a given bin can be mapped to changed luminance values.

[0217] Figure 12 shows a graph representing an exemplary forward mapping. In Figure 12, only five bins are shown as an example.

[0218] Referring to FIG. 12, the x-axis represents input luminance values, and the y-axis represents modified output luminance values. The x-axis is divided into five bins or fragments, and each bin has a length L. That is, the five bins mapped to the modified luminance values ​​have the same length. The forward lookup table (FwdLUT) can be constructed using data available in the tile group header (e.g., reshaper data), from which mapping can be facilitated.

[0219] In one embodiment, output pivot points associated with the bin indices may be calculated. The output pivot points may set (mark) the minimum and maximum boundaries of the output range of the Luma Codeword Reshaping. The process of calculating the output pivot points may be performed based on a piecewise cumulative distribution function of the number of codewords. The output pivot range may be divided based on the maximum number of bins to be used and the size of the lookup table (FwdLUT or InvLUT). As an example, the output pivot range may be divided based on the product between the maximum number of bins and the size of the lookup table. For example, if the product between the maximum number of bins and the size of the lookup table is 1024, the output pivot range may be divided into 1024 entries. The division of the output pivot range may be performed (applied or achieved) based on (using) a scaling factor. In one example, the scaling factor may be derived based on Equation 6 below.

[0220]

[0221] In the above Equation 6, SF represents the scaling factor, and y1 and y2 represent output pivot points corresponding to each bin. Additionally, FP_PREC and c may be predetermined constants. The scaling factor determined based on the above Equation 6 may be referred to as the scaling factor for forward reshaping.

[0222] In another embodiment, regarding inverse reshaping (inverse mapping), for a defined range of bins (e.g., from reshaper_model_min_bin_idx to reshape_model_max_bin_idx), input reshaped pivot points and mapped inverse output pivot points (given as bin index * number of initial codewords) corresponding to the mapped pivot points of the forward lookup table (FwdLUT) are patched. In another example, the scaling factor (SF) can be derived based on Equation 7 below.

[0223]

[0224] In the above Equation 7, SF represents the scaling factor, x1 and x2 represent input pivot points, and y1 and y2 represent output pivot points corresponding to each piece (bin). Here, the input pivot points may be pivot points mapped based on the forward lookup table (FwdLUT), and the output pivot points may be pivot points inverse-mapped based on the inverse lookup table (InvLUT). Additionally, FP_PREC may be a predetermined constant. The FP_PREC in Equation 7 may be the same as or different from the FP_PREC in Equation 6. The scaling factor determined based on the above Equation 4 may be referred to as the scaling factor for inverse reshaping. During inverse reshaping, the splitting of the input pivot points may be performed based on the scaling factor of Equation 4. Based on the divided input pivot points, pivot values ​​corresponding to the minimum and maximum bin values ​​are specified for bin indices in the range from 0 to the minimum bin index (reshaper_model_min_bin_idx) and / or from the minimum bin index (reshaper_model_min_bin_idx) to the maximum bin index (reshape_model_max_bin_idx).

[0225] Table 22 below shows the syntax of a reshaper model according to one embodiment. The reshaper model may be referred to as an LMCS model. Here, the reshaper model is described by way of example as a tile group reshaper, but the present specification is not necessarily limited by this embodiment. For example, the reshaper model may be included in an APS, or the tile group reshaper model may be referred to as a slice reshaper model.

[0226]

[0227] The semantics of the syntax elements included in the syntax of Table 22 above may include, for example, the items disclosed in the following table.

[0228]

[0229] The above reshaper model includes reshape_model_min_bin_idx, reshape_model_delta_max_bin_idx, reshaper_model_bin_delta_abs_cw_prec_minus1, reshape_model_bin_delta_abs_CW[i], and reshaper_model_bin_delta_sign_CW_flag[i] as components. Each of these components will be described in detail below.

[0230] reshape_model_min_bin_idx represents the minimum bin (or fragment) index used during the reshaper configuration process. The value of reshape_model_min_bin_idx can range from 0 to MaxBinIdx. For example, MaxBinIdx can be 15.

[0231] In one embodiment, the tile group reshaper model may first parse two indices (or parameters), reshaper_model_min_bin_idx and reshaper_model_delta_max_bin_idx. Based on these two indices, the maximum bin index (reshaper_model_max_bin_idx) may be derived (determined). reshape_model_delta_max_bin_idx may represent the actual maximum bin index (reshape_model_max_bin_idx) used in the reshaper configuration process subtracted from the allowed maximum bin index MaxBinIdx. The value of the maximum bin index (reshaper_model_max_bin_idx) may range from 0 to MaxBinIdx. For example, MaxBinIdx may be 15. As an example, the value of reshape_model_max_bin_idx may be derived based on Equation 8 below.

[0232]

[0233] The maximum bin index (reshaper_model_max_bin_idx) may be greater than or equal to the minimum bin index (reshaper_model_min_bin_idx). The minimum bin index may be referred to as the minimum allowed bin index or the allowed minimum bin index, and the maximum bin index may also be referred to as the maximum allowed bin index or the allowed maximum bin index.

[0234] If the maximum bin index (rehape_model_max_bin_idx) is derived, the syntax component reshaper_model_bin_delta_abs_cw_prec_minus1 can be parsed. Based on reshaper_model_bin_delta_abs_cw_prec_minus1, the number of bits used to represent the syntax reshape_model_bin_delta_abs_CW[i] can be determined. For example, the number of bits used to represent reshape_model_bin_delta_abs_CW[i] can be equal to reshaper_model_bin_delta_abs_cw_prec_minus1 plus 1.

[0235] reshape_model_bin_delta_abs_CW[i] may represent information related to the absolute delta codeword value (absolute value of the delta codeword) of the i-th bin. In one example, if the absolute delta codeword value of the i-th bin is greater than 0, reshaper_model_bin_delta_sign_CW_flag[i] may be parsed. The sign of reshape_model_bin_delta_abs_CW[i] may be determined based on reshaper_model_bin_delta_sign_CW_flag[i]. In one example, if reshaper_model_bin_delta_sign_CW_flag[i] is 0 (or false), the corresponding variable RspDeltaCW[i] may have a positive sign. Otherwise (if reshaper_model_bin_delta_sign_CW_flag[i] is not 0, or if reshaper_model_bin_delta_sign_CW_flag[i] is 1 (or true), the corresponding variable RspDeltaCW[i] may have a negative sign. If reshape_model_bin_delta_sign_CW_flag[i] does not exist, it may be considered 0 (or false).

[0236] In one embodiment, the variable RspDeltaCW[i] can be derived based on the reshape_model_bin_delta_abs_CW[i] and reshape_model_bin_delta_sign_CW_flag[i] described above. RspDeltaCW[i] can be referred to as the value of a delta codeword. For example, RspDeltaCW[i] can be derived based on Equation 9 below.

[0237]

[0238] In the above mathematical formula 9, reshape_model_bin_delta_sign_CW[i] may be information related to the sign of RspDeltaCW[i]. For example, reshape_model_bin_delta_sign_CW[i] may be the same as the aforementioned reshaper_model_bin_delta_sign_CW_flag[i]. Here, i may be in the range from the minimum bin index (reshaper_model_min_bin_idx) to the maximum bin index (reshape_model_max_bin_idx).

[0239] The variable (or array) RspCW[i] can be derived based on RspDeltaCW[i]. Here, RspCW[i] can represent the number of codewords assigned (distributed) to the i-th bin. That is, the number of codewords assigned (distributed) to each bin can be stored in the form of an array. In one example, if i is less than the aforementioned reshaper_model_min_bin_idx or greater than the reshaper_model_max_bin_idx (i <reshaper_model_min_bin_idx or reshaper_model_max_bin_idx<i), RspCW[i]는 0일 수 있다. 이외의 경우(i가 전술된 reshaper_model_min_bin_idx보다 크거나 같고 그리고 reshaper_model_max_bin_idx보다 작거나 같으면(reshaper_model_min_bin_idx<=i<=reshaper_model_max_bin_idx), RspCW[i]는 상술된 RspDeltaCW[i], 루마 비트 심도(BitDepth Y It can be derived based on ), and / or MaxBinIdx. In this case, for example, RspCW[i] can be derived based on Equation 10 below.

[0240]

[0241] In the above mathematical formula 10, OrgCW may be a predetermined value, for example, determined based on the mathematical formula 11 below.

[0242]

[0243] In the above mathematical formula 11, BitDepth Y is the luma bit depth, and MaxBinIdx represents the maximum allowable bin index. In one example, BitDepth Y If is 10, RspCW[i] can have values ​​from 32 to 2*OrgCW-1.

[0244] The variable InputPivot[i] can be derived based on the OrgCW described above. For example, InputPivot[i] can be derived based on Equation 12 below.

[0245]

[0246] Variables ReshapePivot[i], ScaleCoef[i], and / or InvScaleCoeff[i] can be derived based on the aforementioned RspCW[i], InputPivot[i], and / or OrgCW, for example, ReshapePivot[i], ScaleCoef[i], and / or InvScaleCoeff[i] can be derived based on Table 24 below.

[0247]

[0248] In Table 24 above, a for loop statement in which i increases from 0 to MaxBinIdx may be used, and shiftY may be a predetermined constant for bit shifting. Whether InvScaleCoeff[i] is derived based on RspCW[i] may be determined based on a conditional clause depending on whether RspCW[i] is 0.

[0249] ChromaScaleCoef[i] for deriving the chroma residual scaling factor can be derived based on Table 25 below.

[0250]

[0251] In Table 25 above, shiftC may be a predetermined constant for bit shifting. Referring to Table 25 above, whether ChromaScaleCoef[i] is derived based on the array ChromaResidualScaleLut may be determined based on a conditional clause depending on whether RspCW[i] is 0. Here, ChromaResidualScaleLut may be a predetermined array. However, the array ChromaResidualScaleLut is merely exemplary, and the present embodiment is not necessarily limited by Table 25.

[0252] The method for deriving the i-th variables has been described above. The i+1-th variables can be based on ReshapePivot[i+1], and for example, ReshapePivot[i+1] can be derived based on Equation 13.

[0253]

[0254] In the above mathematical formula 13, RspCW[i] may be derived based on the aforementioned mathematical formulas 10 and / or 11. Luma mapping may be performed based on the embodiments and examples described above, and the syntax described above and the components included therein may be merely exemplary expressions and the embodiments are not limited by the tables or mathematical formulas described above.

[0255] In one embodiment of this document, a signaling structure is proposed for efficiently applying the aforementioned ALF and / or LMCS. According to one embodiment of this document, for example, ALF data and / or LMCS data may be included in an HLS (e.g., APS), and a filter and / or LMCS model (reshaper model) for the ALF can be adaptively derived by signaling a referenced APS ID through header information (e.g., picture header, slice header) which is a lower level of the APS. The LMCS model may be derived based on LMCS parameters. Additionally, for example, multiple APS IDs may be signaled through the header information, and through this, different ALF and / or LMCS models can be applied in block units within the same picture / slice.

[0256] For example, in one embodiment of the present document, an ALF parameter may be conditionally present within a tile group header or a slice header. This may be useful in applications requiring bitstream extraction of a VCL (Video coding Layer) unit. To facilitate such applications, it may be advantageous for the tile group header or the slice header to also have an ALF parameter.

[0257] To this end, the tile group header (syntax) or slice header (syntax) may include a new syntax element. For example, the new syntax element may represent the tile_group_alf_usage_flag syntax element or the slice_alf_usage_flag syntax element. For example, if the new syntax element is parsed and indicates enable, alf_data() may be parsed within the tile group header or slice header. That is, if the value of the tile_group_alf_usage_flag syntax element or the value of the slice_alf_usage_flag syntax element indicates enable, the tile group header or slice header may include alf_data(). Alternatively, for example, if alf_data() is not included in the tile group header or slice header, the APS may be parsed when the ALF tool is enabled at the sequence level and / or for the tile group (or slice). For example, the new syntax element (tile_group_alf_usage_flag syntax element or slice_alf_usage_flag syntax element) may also be called an ALF usage flag.

[0258] In one example, a tile group header or a slice header may include the syntax of Table 26 or Table 27 below. In one example, Table 26 or Table 27 may be part of a tile group header or a slice header.

[0259]

[0260]

[0261] The semantics of the syntax elements included in the syntax of Table 26 or Table 27 above may include, for example, the items disclosed in the following tables.

[0262]

[0263]

[0264] For example, if the ALF usage flag indicates enable (e.g., the value of the tile_group_alf_usage_flag syntax element is 1 or the value of the slice_alf_usage_flag syntax element is 1), alf_data() may be included in the tile group header or slice header, and a filter for ALF may be derived based on this. Or, for example, if the ALF usage flag indicates disable (e.g., the value of the tile_group_alf_usage_flag syntax element is 0 or the value of the slice_alf_usage_flag syntax element is 0), APS index information (e.g., the tile_group_aps_id syntax element or the slice_aps_id syntax element) may be included in the tile group header or slice header, and a filter for ALF may be derived based on the ALF data within the APS pointed to by the APS index information. Here, the APS index information may also be referred to as APS ID information.

[0265] For example, in another embodiment of this document, reshaper parameters may be included in the APS along with ALF parameters. While conventional reshaper parameters were included in tile group headers or slice headers, it may be advantageous to encapsulate the reshaper parameters so that they are parsed together with the ALF data within the APS. Accordingly, in another embodiment, reshaper parameters and ALF parameters may be included together in the APS. Here, the reshaper data may include reshaper parameters, and the reshaper parameters may be referred to as LMCS parameters.

[0266] To this end, ALF and Reshaper tool flags may be evaluated first in the SPS. For example, the decision regarding the enable / disable of the use of ALF and / or Reshaper may be specified by the SPS. Alternatively, said decision may be determined by information or syntax elements included in the SPS. The two tool functions (ALF availability flag and Reshaper availability flag) may operate independently of each other. That is, the two tool functions may be included independently of each other within the SPS. For example, since ALF and Reshaper may be required to construct tables before the VCL NAL unit is decoded, encapsulating ALF data and Reshaper data within the APS may help provide functionality by grouping tools of similar mechanisms of functionality together.

[0267] For example, as described above, the availability of the ALF tool can be determined through the ALF availability flag (e.g., sps_alf_enabled_flag) in the SPS, and subsequently, whether the ALF is available in the current picture or slice can be indicated through the ALF availability flag (e.g., slice_alf_enabled_flag) in the header information. If the value of the ALF availability flag in the header information is 1, the ALF-related APS ID count syntax element can be parsed / signaled. Furthermore, ALF-related APS ID syntax elements equal to the number of ALF-related APS IDs derived based on the ALF-related APS ID count syntax element can be parsed / signaled. That is, this can indicate that multiple APS can be parsed or referenced through a single header information.

[0268] Additionally, the availability of an LMCS (or reshaping) tool can be determined, for example, through an LMCS availability flag in the SPS (e.g., sps_reshaper_enabled_flag). The sps_reshaper_enabled_flag may be referred to as sps_lmcs_enabled_flag. Whether LMCS is available in the current picture or slice can be indicated through an LMCS availability flag in the header information (e.g., slice_lmcs_enabled_flag). If the value of the LMCS availability flag in the header information is 1, an LMCS-related APS ID syntax element can be parsed / signaled. An LMCS model (reshaper model) can be derived from the APS pointed to by the LMCS-related APS ID syntax element. For example, the above APS may further include an LMCS data field, and the LMCS data field may include the above-described LMCS model (reshaper model) information.

[0269] In one example, the SPS may include the syntax of Table 30 below. The syntax of Table 30 may include the sps_alf_enabled_flag syntax element and / or the sps_reshaper_enabled_flag syntax element as tool enabling flags. For example, the sps_alf_enabled_flag syntax element may be used to specify whether ALF is used in a coded video sequence (CVS). The sps_reshaper_enabled_flag syntax element may be used to specify whether a reshaper is used in a CVS. For example, the sps_alf_enabled_flag syntax element may be a flag that enables ALF in the SPS. Additionally, the sps_reshaper_enabled_flag syntax element may be a flag that enables reshaping in the SPS. In one example, the syntax of Table 30 may be part of the SPS.

[0270]

[0271] In one example, the semantics that sps_alf_enabled_flag and sps_reshaper_enabled_flag can represent are as shown in Table 31 below.

[0272]

[0273] Also, in one example, the APS may include the syntax of Table 32 or Table 33 below. In one example, Table 32 or Table 33 may be part of the APS.

[0274]

[0275]

[0276] In Table 32 or Table 33 above, the adaptation_parameter_set_id syntax element and / or the aps_extension_data_flag syntax element may represent the same information as described in the various embodiments described above.

[0277] Also, in one example, the tile group header or slice header may include the syntax of Table 34 or Table 35 below. In one example, Table 34 or Table 35 may be part of the tile group header or slice header.

[0278]

[0279]

[0280] Among the syntax elements included in the syntax of Table 34 or Table 35 above, the semantics of the tile_group_aps_id_alf syntax element, tile_group_aps_id_reshaper syntax element, slice_aps_id_alf syntax element and / or slice_aps_id_reshaper syntax element may include the matters disclosed in the following tables, and other syntax elements may represent the same information as described by the various embodiments described above.

[0281]

[0282]

[0283] Alternatively, the same APS may be used to parse information about the reshaper model (e.g., tile_group_reshaper_model()) as well as ALF data (e.g., alf_data()). In this case, the tile group header or slice header may contain the syntax of Table 38 or Table 39 below. In one example, Table 38 or Table 39 may be part of the tile group header or slice header.

[0284]

[0285]

[0286] The syntax elements included in the syntax of Table 38 or Table 39 above may represent the same information as described by the various embodiments described above. Here, a common tile_group_aps_id syntax element or a slice_aps_id syntax element may be parsed to access the ALF and reshaper models in a single APS.

[0287] The tile group header or slice header according to Table 34 or Table 35 above may include APS index information for ALF data (e.g., tile_group_aps_id_alf syntax element or slice_aps_id_alf syntax element) and APS index information for reshaper data (e.g., tile_group_aps_id_reshaper syntax element or slice_aps_id_reshaper syntax element) separately.

[0288] A tile group header or slice header according to Table 38 or Table 39 above includes one APS index information (e.g., a tile_group_aps_id syntax element or a slice_aps_id syntax element), and said one APS index information can be used as both APS index information for ALF data and APS index information for reshaper data.

[0289] For example, ALF and reshaper can reference different APS. In other words, in Table 34 or Table 35, the APS index information for ALF data (e.g., the tile_group_aps_id_alf syntax element or the slice_aps_id_alf syntax element) and the APS index information for reshaper data (e.g., the tile_group_aps_id_reshaper syntax element or the slice_aps_id_reshaper syntax element) may represent index information for different APS.

[0290] Or, for example, the ALF and the reshaper may reference the same APS. In other words, in Table 34 or Table 35, the APS index information for the ALF data (e.g., the tile_group_aps_id_alf syntax element or the slice_aps_id_alf syntax element) and the APS index information for the reshaper data (e.g., the tile_group_aps_id_reshaper syntax element or the slice_aps_id_reshaper syntax element) may represent index information for the same APS. Or, if the ALF and the reshaper reference the same APS, a single APS index information (e.g., the tile_group_aps_id syntax element or the slice_aps_id syntax element) may be included, as in Table 38 or Table 39.

[0291] For example, in another embodiment of this document, reshaper parameters may be conditionally included in the APS along with ALF parameters. To this end, the tile group header or slice header may reference one or more APS that can be operated upon the application's requirements. For example, use cases regarding sub-bitstream extraction processes / bitstream splicing may be considered. Previously, the non-indication of bitstream attributes could imply system constraints. In particular, the system used a single SPS for the whole (thus efficiently splicing complex CVSs from different encoding devices), or announced all SPS at the start of the session (thus reducing the flexibility of encoding devices in run-time adjustments). Therefore, depending on the application, it may be advantageous to parse the necessary ALF data and / or reshaper model data within the tile group header or slice header. In this case, the system may have the flexibility to extract from its own VCL NAL units for processing. Additionally, the NAL unit may benefit from signaling information within the tile group header or slice header, in addition to the flexibility provided by signaling the APS ID.

[0292] To this end, in one example, when a NAL unit needs to parse ALF data (e.g., alf_data()) and / or a reshaper model (e.g., tile_group_reshaper_model() or slice_reshaper_model()) or refer to an APS instead, the tile_group_alf_reshaper_usage_flag syntax element or the slice_alf_reshaper_usage_flag syntax element may be used. That is, the tile_group_alf_reshaper_usage_flag syntax element or the slice_alf_reshaper_usage_flag syntax element may be included in the tile group header or the slice header.

[0293] In one example, a tile group header or slice header may include the syntax of Table 40 or Table 41 below. In one example, Table 40 or Table 41 may be part of a tile group header or slice header.

[0294]

[0295]

[0296] Among the syntax elements included in the syntax of Table 40 or Table 41 above, the semantics of the tile_group_alf_reshaper_usage_flag syntax element or the slice_alf_reshaper_usage_flag syntax element may include the matters disclosed in the following tables, and other syntax elements may represent the same information as described by the various embodiments described above.

[0297]

[0298]

[0299] For example, the tile_group_alf_reshaper_usage_flag syntax element or the slice_alf_reshaper_usage_flag syntax element may indicate whether to use an APS ID. That is, if the value of the tile_group_alf_reshaper_usage_flag syntax element or the slice_alf_reshaper_usage_flag syntax element is 1, the APS ID is not used, and accordingly, the APS may not be referenced. Or, if the value of the tile_group_alf_reshaper_usage_flag syntax element or the slice_alf_reshaper_usage_flag syntax element is 0, the APS ID may be used, and the APS ID may be specified by the tile_group_aps_pic_parameter_set syntax element or the slice_aps_pic_parameter_set syntax element. Here, the tile_group_alf_reshaper_usage_flag syntax element or the slice_alf_reshaper_usage_flag syntax element may also be referred to as the APS usage flag or the ALF & reshaper usage flag.

[0300] Alternatively, for example, a single APS may be referenced to access both parameters, providing flexibility to have both ALF and reshaper model parameters within a tile group header or slice header on the same tile. In this case, the tile group header or slice header may include the syntax of Table 44 or Table 45 below. In one example, Table 44 or Table 45 may be part of the tile group header or slice header.

[0301]

[0302]

[0303] The tile group header or slice header according to Table 40 or Table 41 above may include APS index information for ALF data (e.g., tile_group_aps_id_alf syntax element or slice_aps_id_alf syntax element) and APS index information for reshaper data (e.g., tile_group_aps_id_reshaper syntax element or slice_aps_id_reshaper syntax element) separately.

[0304] A tile group header or slice header according to Table 44 or Table 45 above includes one APS index information (e.g., a tile_group_aps_id syntax element or a slice_aps_id syntax element), and said one APS index information can be used as both APS index information for ALF data and APS index information for reshaper data.

[0305] For example, ALF and reshaper can reference different APS. In other words, in Table 40 or Table 41, the APS index information for ALF data (e.g., the tile_group_aps_id_alf syntax element or the slice_aps_id_alf syntax element) and the APS index information for reshaper data (e.g., the tile_group_aps_id_reshaper syntax element or the slice_aps_id_reshaper syntax element) may represent index information for different APS.

[0306] Or, for example, the ALF and the reshaper may reference the same APS. In other words, the APS index information for the ALF data in Table 40 or Table 41 (e.g., the tile_group_aps_id_alf syntax element or the slice_aps_id_alf syntax element) and the APS index information for the reshaper data (e.g., the tile_group_aps_id_reshaper syntax element or the slice_aps_id_reshaper syntax element) may represent index information for the same APS. Or, if the ALF and the reshaper reference the same APS, a single APS index information (e.g., the tile_group_aps_id syntax element or the slice_aps_id syntax element) may be included, as in Table 44 or Table 45.

[0307] FIGS. 13 and 14 schematically illustrate an example of a video / image encoding method and related components according to the embodiment(s) of the present document. The method disclosed in FIG. 13 may be performed by the encoding device disclosed in FIG. 2. Specifically, for example, S1300 of FIG. 13 may be performed by the adder (250) of the encoding device, S1310 of FIG. 13 may be performed by the filtering unit (260) of the encoding device, and S1320 of FIG. 13 may be performed by the entropy encoding unit (240) of the encoding device. The method disclosed in FIG. 13 may include the embodiments described above in the present document.

[0308] Referring to FIG. 13, the encoding device generates restoration samples of the current block within the current picture (S1300). For example, the encoding device may derive predicted samples of the current block based on a prediction mode. In this case, various prediction methods disclosed herein, such as inter-prediction or intra-prediction, may be applied. Residual samples may be derived based on the predicted samples and original samples. In this case, residual information may be derived based on the residual samples. (Modified) residual samples may be derived based on the residual information. Restoration samples may be generated based on the (modified) residual samples and the predicted samples. A restored block and a restored picture may be derived based on the restoration samples.

[0309] The encoding device generates reshaping-related information for the restored samples (S1310). Alternatively, for example, the encoding device may generate reshaping-related information and / or ALF-related information for the restored samples. For example, the encoding device may derive parameters related to reshaping that can be applied for filtering the restored samples and generate reshaping-related information. Alternatively, for example, the encoding device may derive parameters related to ALF that can be applied for filtering the restored samples and generate ALF-related information. For example, the reshaping-related information may include at least some of the reshaping-related information described above in this document. Or, for example, the ALF-related information may include at least some of the ALF-related information described above in this document. Here, the reshaping-related information may be represented as LMCS-related information, and the reshaping-related parameters may be represented as LMCS-related parameters. For example, reshaping data may be called LMCS data, parameters related to reshaping, reshaping model information, or information included in the reshaping model.

[0310] The encoding device encodes image information including information regarding the generation of restoration samples and information regarding reshaping (S1320). Alternatively, for example, the encoding device may encode image information including at least some of information regarding the generation of restoration samples, information regarding reshaping, or information regarding ALF. Here, the information regarding reshaping may be referred to as LMCS information. The image information may also be referred to as video information. For example, the information for generating the restoration samples may include, for example, prediction information and / or residual information. The prediction information may include information regarding various prediction modes (e.g., merge mode, MVP mode, etc.), MVD information, etc.

[0311] The above image information may include various information according to embodiments of the present document. For example, the above image information may include at least one piece of information or at least one syntax element disclosed in at least one of Tables 1 to 45 described above.

[0312] For example, the image information may include the adaptation parameter set (APS) and / or ID information for the APS. Or, for example, the reshaping-related information may include the APS and / or ID information for the APS. For example, the APS may include reshaping data. For example, the reshaping data may include information for deriving a mapping index representing the mapping relationship of the values ​​of the (luminance component) restored samples of the current block. Or, for example, the reshaping data may include information for performing a reshaping procedure. For example, the reshaping procedure may represent an inverse reshaping procedure, and the mapping index may include an index used for the inverse reshaping procedure. However, the reshaping procedure according to this document is not limited to an inverse reshaping procedure, and a forward reshaping procedure or a chroma reshaping procedure may be used. However, if a forward reshaping procedure or a chroma reshaping procedure is used, the order in which reshaping-related information is generated within the encoding procedure may differ, and the index may also differ depending on the forward reshaping procedure or the chroma reshaping procedure. For example, the reshaping procedure may be referred to as the LMCS procedure, the reshaping data as LMCS data, and the reshaping-related information as LMCS-related information, and the same may apply below. Alternatively, for example, the ALF-related information may include the APS and / or ID information for the APS. For example, the APS may include ALF data. For example, the ALF data may include information for deriving ALF filter coefficients. Alternatively, for example, the ALF data may include information for performing an ALF procedure.

[0313] For example, the reshaping data may be represented based on ID information for the APS. Or, the ALF data may be represented based on ID information for the APS. That is, the reshaping data may be included in the APS pointed to by the ID information for the APS. Or, the ALF data may be included in the APS pointed to by the ID information for the APS.

[0314] Alternatively, for example, the reshaping data and the ALF data may be included in a single APS. In this case, since the ID information for the APS for the reshaping data and the ID information for the APS for the ALF data may be the same, they may be distinguished and represented separately, but they may also be represented as a single unit. In this case, for convenience of explanation, the single APS containing the reshaping data and the ALF data may be referred to as the first APS.

[0315] Alternatively, for example, the reshaping data and the ALF data may be included in different APSs. In this case, for convenience of explanation, the APS containing the reshaping data may be referred to as the first APS, and the APS containing the ALF data may be referred to as the second APS. In this case, the first APS is different from the second APS, and the ID information for the first APS may also be different from the ID information for the second APS.

[0316] Alternatively, for example, the reshaping data or the ALF data may be included in a single APS. In this case, for convenience of explanation, the APS containing the reshaping data or the ALF data may be referred to as the first APS.

[0317] For example, if the reshaping data and the ALF data are included in a single APS and ID information is separately distinguished, included in different APS, or if either the reshaping data or the ALF data is included in the APS, the ID information for the APS for the reshaping data can be represented by the tile_group_aps_id_reshaper syntax element or the slice_aps_id_reshaper syntax element. Alternatively, the ID information for the APS for the ALF data can be represented by the tile_group_aps_id_alf syntax element or the slice_aps_id_alf syntax element. Alternatively, for example, if the reshaping data and the ALF data are included in a single APS, or if either the reshaping data or the ALF data is included in the APS, the ID information for the APS can be represented in the same form, such as the tile_group_aps_id syntax element or the slice_aps_id syntax element. However, for example, ID information for an APS for reshaping and an ALF may be represented separately using ID information for an APS of the same form. For example, ID information for an APS for reshaping and / or ALF may be included in header information. For example, the image information may include the header information, and the header information may include a picture header or a slice header (or tile group header), and the same may be true below.

[0318] For example, the image information may include a Sequence Parameter Set (SPS), and the SPS may include a first reshaping availability flag indicating whether the reshaping is available and / or a first ALF availability flag indicating whether the ALF is available. For example, the first reshaping availability flag may represent the sps_reshaper_enabled_flag syntax element, and the first ALF availability flag may represent the sps_alf_enabled_flag syntax element.

[0319] For example, based on the first reshaping availability flag having a value of 1, the header information may include a second reshaping availability flag indicating whether the reshaping is available in the picture or slice. That is, when the value of the first reshaping availability flag is 1, the header information may include a second reshaping availability flag indicating whether the reshaping is available in the picture or slice. Alternatively, based on the first reshaping availability flag having a value of 0, the header information may not include the second reshaping availability flag. That is, when the value of the first reshaping availability flag is 0, the header information may not include the second reshaping availability flag. For example, the second reshaping availability flag may represent the tile_group_reshaper_enabled_flag syntax element or the slice_reshaper_enabled_flag syntax element.

[0320] Alternatively, for example, based on the first reshaping availability flag having a value of 1, the header information may include a reshaping model present flag indicating whether a reshaping model exists in the picture or slice. That is, when the value of the first reshaping availability flag is 1, the header information may include the reshaping model present flag. For example, the reshaping model present flag may represent the tile_group_reshaper_model_present_flag syntax element or the slice_reshaper_model_present_flag syntax element. Also, for example, based on the reshaping model present flag having a value of 1, the header information may include the second reshaping availability flag. That is, when the value of the reshaping model present flag is 1, the header information may include the second reshaping availability flag.

[0321] For example, the performance of the reshaping procedure may be indicated based on the first reshaping availability flag, the reshaping model existence flag and / or the second reshaping availability flag. Or, for example, the performance of the reshaping procedure may be indicated based on the first reshaping availability flag with a value of 1, the reshaping model existence flag with a value of 1 and / or the second reshaping availability flag with a value of 1.

[0322] For example, based on the first ALF availability flag having a value of 1, the header information may include a second ALF availability flag indicating whether the ALF is available in the picture or slice. That is, when the value of the first ALF availability flag is 1, the header information may include a second ALF availability flag indicating whether the ALF is available in the picture or slice. Alternatively, based on the first ALF availability flag having a value of 0, the header information may not include the second ALF availability flag. That is, when the value of the first ALF availability flag is 0, the header information may not include the second ALF availability flag. For example, the second ALF availability flag may represent the tile_group_alf_enabled_flag syntax element or the slice_alf_enabled_flag syntax element.

[0323] For example, based on the second ALF availability flag having a value of 1, the header information may include an ALF usage flag indicating whether the ALF is used in the picture or slice. That is, when the value of the second ALF availability flag is 1, the header information may include an ALF usage flag indicating whether the ALF is used in the picture or slice. Alternatively, based on the second ALF availability flag having a value of 0, the header information may not include the ALF usage flag. That is, when the value of the second ALF availability flag is 0, the header information may not include the ALF usage flag. For example, the ALF usage flag may represent the tile_group_alf_usage_flag syntax element or the slice_alf_usage_flag syntax element.

[0324] For example, based on the ALF usage flag having a value of 1, the header information may include ALF data. That is, when the value of the ALF usage flag is 1, the header information may include ALF data. In this case, ALF data may be displayed without using the APS represented by the ID information for the APS. Or, for example, based on the ALF usage flag having a value of 0, the header information may include ID information for the APS.

[0325] For example, based on the first reshaping availability flag with a value of 1 and the first ALF availability flag with a value of 1 (i.e., when the value of the first reshaping availability flag is 1 and the value of the first ALF availability flag is 1), the header information may include a second ALF availability flag indicating whether the ALF is available in the picture or slice, and based on the second ALF availability flag with a value of 1 (i.e., when the value of the second ALF availability flag is 1), the header information may include an ALF and reshaping usage flag indicating whether the ALF and the reshaping are used in the picture or slice. For example, the ALF and reshaping usage flag may represent the tile_group_alf_reshaper_usage_flag syntax element or the slice_alf_reshaper_usage_flag syntax element.

[0326] For example, based on the ALF and reshaping usage flag having a value of 1 (i.e., when the value of the ALF and reshaping usage flag is 1), the header information may include the ALF data and the reshaping data. Or, for example, based on the ALF and reshaping usage flag having a value of 0 (i.e., when the value of the ALF and reshaping usage flag is 0), the header information may include ID information for the first APS and ID information for the second APS. Here, the ID information for the first APS may represent ID information for the APS for reshaping data, and the ID information for the second APS may represent ID information for the APS for ALF data. For example, the first APS and the second APS may be the same, but may also be different.

[0327] For example, an encoding device may generate a bitstream or encoded information by encoding image information containing all or part of the information (or syntax elements) described above. Alternatively, it may output it in the form of a bitstream. Furthermore, the bitstream or encoded information may be transmitted to a decoding device via a network or a storage medium. Alternatively, the bitstream or encoded information may be stored on a computer-readable storage medium, and the bitstream or encoded information may be generated by the image encoding method described above.

[0328] FIGS. 15 and 16 schematically illustrate an example of an image / video decoding method and related components according to an embodiment of the present document. The method disclosed in FIG. 15 may be performed by the decoding device disclosed in FIG. 3. Specifically, for example, S1500 of FIG. 15 may be performed by the entropy decoding unit (310) of the decoding device, S1510 may be performed by the adder (340) of the decoding device, and S1520 may be performed by the filtering unit (350) of the decoding device. The method disclosed in FIG. 15 may include the embodiments described above in the present document.

[0329] Referring to FIG. 15, a decoding device receives / acquires image information through a bitstream (S1500). The image information may include various information according to an embodiment of the present document. For example, the image information may include at least one piece of information or at least one syntax element disclosed in at least one of Tables 1 to 45 described above. For example, the image information may be referred to as video information. For example, the image information may include at least some of information regarding the generation of restoration samples, information related to reshaping, or information related to ALF. For example, the information for generating the restoration samples may include, for example, prediction-related information and / or residual information. The prediction-related information may include information regarding various prediction modes (e.g., merge mode, MVP mode, etc.), MVD information, etc.

[0330] The decoding device generates restoration samples of the current block based on the image information (S1510). For example, the decoding device may derive prediction samples of the current block based on prediction-related information included in the image information. Or, for example, the decoding device may derive residual samples based on residual information included in the image information. Or, for example, the decoding device may generate restoration samples based on the prediction samples and the residual samples. A restoration block and a restoration picture may be derived based on the restoration samples.

[0331] The decoding device performs a reshaping procedure for the restored samples (S1520). Alternatively, for example, the decoding device may perform a reshaping procedure and / or an ALF procedure for the restored samples. For example, the decoding device may obtain reshaping-related information and / or ALF-related information from the image information, derive parameters related to reshaping and / or parameters related to ALF based thereon, and perform a reshaping procedure or an ALF procedure based thereon. For example, the reshaping-related information may include at least some of the reshaping-related information described above in this document. Or, for example, the ALF-related information may include at least some of the ALF-related information described above in this document. Here, the reshaping-related information may be represented as LMCS-related information, and the reshaping-related parameters may be represented as LMCS-related parameters. For example, reshaping data may be called LMCS data, parameters related to reshaping, reshaping model information, or information included in the reshaping model.

[0332] For example, the image information may include the adaptation parameter set (APS) and / or ID information for the APS. For example, the APS may include reshaping data, and a reshaping procedure may be performed based on the reshaping data. Here, the reshaping procedure may include a procedure for deriving the value of a mapped reshaping sample based on the value of the (luminance component) reshaping sample of the current block and a mapping index. For example, the mapping index may be derived based on the reshaping data. For example, the reshaping procedure may represent an inverse reshaping procedure, and the mapping index may include an index used for the inverse reshaping procedure. However, the reshaping procedure according to this document is not limited to an inverse reshaping procedure, and a forward reshaping procedure or a chroma reshaping procedure may be used. However, if a forward reshaping procedure or a chroma reshaping procedure is used, the order in which the reshaping procedure is performed within the decoding procedure may differ, and the index may also differ depending on the forward reshaping procedure or the chroma reshaping procedure. For example, the reshaping procedure may be represented as the LMCS procedure, the reshaping data may be represented as LMCS data, and the reshaping-related information may be represented as LMCS-related information, and the same may apply below. Alternatively, for example, the above APS may include ALF data, and an ALF procedure may be performed based on the above ALF data. Here, the ALF procedure may include a procedure for deriving ALF filter coefficients based on the above ALF data.

[0333] For example, the reshaping data may be derived based on ID information for the APS. Alternatively, the ALF data may be derived based on ID information for the APS. That is, the reshaping data may be included in the APS pointed to by the ID information for the APS. Alternatively, the ALF data may be included in the APS pointed to by the ID information for the APS.

[0334] Alternatively, for example, the reshaping data and the ALF data may be included in a single APS. In this case, since the ID information for the APS for the reshaping data and the ID information for the APS for the ALF data may be the same, they may be distinguished separately but may also be used as one. In this case, for convenience of explanation, the single APS containing the reshaping data and the ALF data may be referred to as the first APS.

[0335] Alternatively, for example, the reshaping data and the ALF data may be included in different APSs. In this case, for convenience of explanation, the APS containing the reshaping data may be referred to as the first APS, and the APS containing the ALF data may be referred to as the second APS. In this case, the first APS is different from the second APS, and the ID information for the first APS may also be different from the ID information for the second APS.

[0336] Alternatively, for example, the reshaping data or the ALF data may be included in a single APS. In this case, for convenience of explanation, the APS containing the reshaping data or the ALF data may be referred to as the first APS.

[0337] For example, if the reshaping data and the ALF data are included in a single APS and ID information is separately distinguished, included in different APS, or if either the reshaping data or the ALF data is included in the APS, the ID information for the APS for the reshaping data can be represented by the tile_group_aps_id_reshaper syntax element or the slice_aps_id_reshaper syntax element. Alternatively, the ID information for the APS for the ALF data can be represented by the tile_group_aps_id_alf syntax element or the slice_aps_id_alf syntax element. Alternatively, for example, if the reshaping data and the ALF data are included in a single APS, or if either the reshaping data or the ALF data is included in the APS, the ID information for the APS can be represented in the same form, such as the tile_group_aps_id syntax element or the slice_aps_id syntax element. However, for example, ID information for an APS for reshaping and an ALF may be represented separately using ID information for an APS of the same form. For example, ID information for an APS for reshaping and / or ALF may be included in header information. For example, the image information may include the header information, and the header information may include a picture header or a slice header (or tile group header), and the same may be true below.

[0338] For example, the image information may include a Sequence Parameter Set (SPS), and the SPS may include a first reshaping availability flag indicating whether the reshaping is available and / or a first ALF availability flag indicating whether the ALF is available. For example, the first reshaping availability flag may represent the sps_reshaper_enabled_flag syntax element, and the first ALF availability flag may represent the sps_alf_enabled_flag syntax element.

[0339] For example, based on the first reshaping availability flag having a value of 1, the header information may include a second reshaping availability flag indicating whether the reshaping is available in the picture or slice. That is, when the value of the first reshaping availability flag is 1, the header information may include a second reshaping availability flag indicating whether the reshaping is available in the picture or slice. Alternatively, based on the first reshaping availability flag having a value of 0, the header information may not include the second reshaping availability flag. That is, when the value of the first reshaping availability flag is 0, the header information may not include the second reshaping availability flag. For example, the second reshaping availability flag may represent the tile_group_reshaper_enabled_flag syntax element or the slice_reshaper_enabled_flag syntax element.

[0340] Alternatively, for example, based on the first reshaping availability flag having a value of 1, the header information may include a reshaping model present flag indicating whether a reshaping model exists in the picture or slice. That is, when the value of the first reshaping availability flag is 1, the header information may include the reshaping model present flag. For example, the reshaping model present flag may represent the tile_group_reshaper_model_present_flag syntax element or the slice_reshaper_model_present_flag syntax element. Also, for example, based on the reshaping model present flag having a value of 1, the header information may include the second reshaping availability flag. That is, when the value of the reshaping model present flag is 1, the header information may include the second reshaping availability flag.

[0341] For example, the performance of the reshaping procedure may be indicated based on the first reshaping availability flag, the reshaping model existence flag and / or the second reshaping availability flag. Or, for example, the performance of the reshaping procedure may be indicated based on the first reshaping availability flag with a value of 1, the reshaping model existence flag with a value of 1 and / or the second reshaping availability flag with a value of 1.

[0342] For example, based on the first ALF availability flag having a value of 1, the header information may include a second ALF availability flag indicating whether the ALF is available in the picture or slice. That is, when the value of the first ALF availability flag is 1, the header information may include a second ALF availability flag indicating whether the ALF is available in the picture or slice. Alternatively, based on the first ALF availability flag having a value of 0, the header information may not include the second ALF availability flag. That is, when the value of the first ALF availability flag is 0, the header information may not include the second ALF availability flag. For example, the second ALF availability flag may represent the tile_group_alf_enabled_flag syntax element or the slice_alf_enabled_flag syntax element.

[0343] For example, based on the second ALF availability flag having a value of 1, the header information may include an ALF usage flag indicating whether the ALF is used in the picture or slice. That is, when the value of the second ALF availability flag is 1, the header information may include an ALF usage flag indicating whether the ALF is used in the picture or slice. Alternatively, based on the second ALF availability flag having a value of 0, the header information may not include the ALF usage flag. That is, when the value of the second ALF availability flag is 0, the header information may not include the ALF usage flag. For example, the ALF usage flag may represent the tile_group_alf_usage_flag syntax element or the slice_alf_usage_flag syntax element.

[0344] For example, based on the ALF usage flag having a value of 1, the header information may include ALF data. That is, when the value of the ALF usage flag is 1, the header information may include ALF data. In this case, ALF data can be derived without using the APS represented by the ID information for the APS, and an ALF procedure can be performed based thereon. Alternatively, for example, based on the ALF usage flag having a value of 0, the header information may include ID information for the APS.

[0345] For example, based on the first reshaping availability flag with a value of 1 and the first ALF availability flag with a value of 1 (i.e., when the value of the first reshaping availability flag is 1 and the value of the first ALF availability flag is 1), the header information may include a second ALF availability flag indicating whether the ALF is available in the picture or slice, and based on the second ALF availability flag with a value of 1 (i.e., when the value of the second ALF availability flag is 1), the header information may include an ALF and reshaping usage flag indicating whether the ALF and the reshaping are used in the picture or slice. For example, the ALF and reshaping usage flag may represent the tile_group_alf_reshaper_usage_flag syntax element or the slice_alf_reshaper_usage_flag syntax element.

[0346] For example, based on the ALF and reshaping usage flag having a value of 1 (i.e., when the value of the ALF and reshaping usage flag is 1), the header information may include the ALF data and the reshaping data. Or, for example, based on the ALF and reshaping usage flag having a value of 0 (i.e., when the value of the ALF and reshaping usage flag is 0), the header information may include ID information for the first APS and ID information for the second APS. Here, the ID information for the first APS may represent ID information for the APS for reshaping data, and the ID information for the second APS may represent ID information for the APS for ALF data. For example, the first APS and the second APS may be the same, but may also be different.

[0347] For example, a decoding device can decode a bitstream or encoded information to obtain image information containing all or part of the information (or syntax elements) described above. Additionally, the bitstream or encoded information may be stored in a computer-readable storage medium and may cause the decoding method described above to be performed.

[0348] In the embodiments described above, methods are described based on flowcharts as a series of steps or blocks; however, the embodiments are not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps as described above. Furthermore, those skilled in the art will understand that the steps shown in the flowcharts are not exclusive, and other steps may be included, or one or more steps of the flowcharts may be omitted without affecting the scope of the embodiments of this document.

[0349] The method according to the embodiments of the present document described above may be implemented in the form of software, and the encoding device and / or decoding device according to the present document may be included in a device that performs image processing, such as a TV, computer, smartphone, set-top box, display device, etc.

[0350] When the embodiments described in this document are implemented in software, the method described above may be implemented as a module (process, function, etc.) that performs the function described above. The module may be stored in memory and executed by a processor. The memory may be located inside or outside the processor and may be connected to the processor by various well-known means. The processor may include an application-specific integrated circuit (ASIC), other chipsets, logic circuits, and / or data processing devices. The memory may include read-only memory (ROM), random access memory (RAM), flash memory, memory cards, storage media, and / or other storage devices. That is, the embodiments described in this document may be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units illustrated in each figure may be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, information on instructions or algorithms for implementation may be stored on a digital storage medium.

[0351] In addition, the decoding device and encoding device to which the embodiment(s) of this document apply may be included in multimedia broadcasting transmission and reception devices, mobile communication terminals, home cinema video devices, digital cinema video devices, surveillance cameras, video conversation devices, real-time communication devices such as video communication, mobile streaming devices, storage media, camcorders, Video on Demand (VoD) service providers, Over-the-top video (OTT) devices, internet streaming service providers, 3D video devices, virtual reality (VR) devices, augmented reality (AR) devices, video phone video devices, transportation terminals (e.g., vehicle terminals (including autonomous vehicles), airplane terminals, ship terminals, etc.), and medical video devices, and may be used to process video signals or data signals. For example, Over-the-top video (OTT) devices may include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, Digital Video Recorders (DVRs), etc.

[0352] Additionally, the processing method to which the embodiment(s) of this document are applied may be produced in the form of a program that is executed by a computer and may be stored on a computer-readable recording medium. Multimedia data having a data structure according to the embodiment(s) of this document may also be stored on a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distributed storage devices in which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray disc (BD), a Universal Serial Bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. Additionally, the computer-readable recording medium includes a medium implemented in the form of a carrier wave (e.g., transmission over the Internet). Additionally, a bitstream generated by an encoding method may be stored on a computer-readable recording medium or transmitted via a wired or wireless communication network.

[0353] Additionally, the embodiment(s) of this document may be implemented as a computer program product by program code, and said program code may be executed on a computer by the embodiment(s) of this document. said program code may be stored on a computer-readable carrier.

[0354] FIG. 17 shows an example of a content streaming system to which the embodiments disclosed in this document can be applied.

[0355] Referring to FIG. 17, a content streaming system to which embodiments of the present document are applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0356] The above encoding server compresses content input from multimedia input devices, such as smartphones, cameras, and camcorders, into digital data to generate a bitstream and transmits it to the streaming server. As another example, if multimedia input devices, such as smartphones, cameras, and camcorders, generate the bitstream directly, the encoding server may be omitted.

[0357] The bitstream above may be generated by an encoding method or a bitstream generation method to which the embodiments of the present document are applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0358] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server acts as a medium to inform the user of available services. When a user requests a desired service from the web server, the web server transmits it to the streaming server, and the streaming server transmits the multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server plays the role of controlling commands and responses between each device within the content streaming system.

[0359] The streaming server may receive content from a media storage and / or an encoding server. For example, when receiving content from the encoding server, the content may be received in real time. In this case, to provide a seamless streaming service, the streaming server may store the bitstream for a certain period of time.

[0360] Examples of the above user devices may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs (head-mounted displays)), digital TVs, desktop computers, digital signage, etc.

[0361] Each server within the above-mentioned content streaming system can be operated as a distributed server, and in this case, data received from each server can be processed in a distributed manner.

[0362] The claims described in this specification may be combined in various ways. For example, the technical features of the method claims in this specification may be combined to be implemented as a device, and the technical features of the device claims in this specification may be combined to be implemented as a method. Furthermore, the technical features of the method claims and the technical features of the device claims in this specification may be combined to be implemented as a device, and the technical features of the method claims and the technical features of the device claims in this specification may be combined to be implemented as a method.

Claims

Claim 1 In a decoding device that performs image decoding, the decoding device comprises: a memory; and includes at least one processor connected to the memory, wherein the at least one processor: receives image information including an Adaptation Parameter Set (APS) and ID information for the APS via a bitstream; generates reconstructed samples of a current block based on the image information; performs an inverse mapping procedure for the reconstructed samples; performs an Adaptive Loop Filtering (ALF) procedure for the reconstructed samples; the image information includes a Sequence Parameter Set (SPS), the SPS includes an LMCS availability flag indicating the availability of LMCS (Luma Mapping with Chroma Scaling) and an ALF availability flag indicating the availability of ALF; the inverse mapping procedure includes a procedure for deriving the value of a mapped reconstructed sample based on the value of the reconstructed sample of the current block and a mapping index; the APS includes a first APS including LMCS data and a second APS including ALF data; the ID information for the APS includes ID information for the first APS and ID information for the second APS; and the LMCS data based on the ID information for the first APS An image decoding device characterized in that the mapping index is derived based on the LMCS data, the ALF data is derived based on ID information for the second APS, the ALF procedure includes a process of deriving ALF filter coefficients based on the ALF data, and the ID information for the second APS is different from the ID information for the first APS. Claim 2 In an encoding device that performs video encoding, the encoding device comprises: a memory; and includes at least one processor connected to the memory, wherein the at least one processor: generates recovery samples of a current block, generates information related to Luma Mapping with Chroma Scaling (LMCS) for the recovery samples, generates information related to Adaptive Loop Filtering (ALF) for the recovery samples, encodes image information including the generation information of the recovery samples, the LMCS related information, and the ALF related information, wherein the image information includes a sequence parameter set (SPS), wherein the SPS includes an LMCS availability flag indicating the availability of Luma Mapping with Chroma Scaling (LMCS) and an ALF availability flag indicating the availability of ALF, wherein the LMCS related information includes a first Adaptation Parameter Set (APS) containing LMCS data and ID information for the first APS, wherein the LMCS data includes information for deriving a mapping index indicating the mapping relationship of the values ​​of the recovery samples of the current block, and the ALF related information includes a second Adaptation Parameter Set (APS) containing ALF data and the second An image encoding device characterized by including ID information for an APS, wherein the ALF data includes information for deriving ALF filter coefficients, and the ID information for the second APS is different from the ID information for the first APS. Claim 3 In a device for transmitting data for an image, the device comprises at least one processor configured to generate reconstructed samples of a current block, generate Luma Mapping with Chroma Scaling (LMCS) related information for the reconstructed samples, generate Adaptive Loop Filtering (ALF) related information for the reconstructed samples, and obtain a bitstream generated by encoding image information including the generation information of the reconstructed samples, the LMCS related information, and the ALF related information; A video data transmission device comprising: a transmission unit configured to transmit the data including the bitstream; wherein the video information includes a sequence parameter set (SPS), the SPS includes an LMCS availability flag indicating the availability of LMCS (Luma Mapping with Chroma Scaling) and an ALF availability flag indicating the availability of ALF, the LMCS-related information includes a first APS (Adaptation Parameter Set) including LMCS data and ID information for the first APS, the LMCS data includes information for deriving a mapping index indicating the mapping relationship of the values ​​of the restored samples of the current block, the ALF-related information includes a second APS (Adaptation Parameter Set) including ALF data and ID information for the second APS, the ALF data includes information for deriving ALF filter coefficients, and the ID information for the second APS is different from the ID information for the first APS.

Citation Information

Patent Citations

  • Aps signaling-based video or image coding

    KR1020220057661A