Video or image coding based on adaptive loop filter

Abstract

According to the disclosure of this document, ALF parameters and / or LMCS parameters may be signaled hierarchically, thereby achieving a reduction in the amount of data to be signaled for video / image coding and an increase in coding efficiency.

Description

Technical Field

[0001] The present disclosure relates to image or video coding based on adaptive loop filters. Background Art

[0002] Recently, there is an increasing demand for high-resolution, high-quality images / videos such as 4K or 8K ultra-high definition (UHD) images / videos in various fields. As the image / video resolution or quality becomes higher, relatively more information or bits are transmitted compared to conventional image / video data. Therefore, if the image / video data is transmitted via a medium such as an existing wired / wireless broadband line or is stored in a conventional storage medium, the cost of transmission and storage is easily increased.

[0003] In addition, there is growing interest and demand for virtual reality (VR) and artificial reality (AR) content and immersive media such as holograms; and there is also growing broadcasting of images / videos that exhibit image / video characteristics different from actual images / videos (e.g., game images / videos).

[0004] Therefore, highly efficient image / video compression technology is required to effectively compress and transmit, store, or play high-resolution, high-quality images / videos showing various characteristics as described above.

[0005] In addition, there is discussion on techniques such as utilizing Luma Mapping and Chroma Scaling (LMCS) and Adaptive Loop Filtering (ALF) to improve compression efficiency and increase subjective / objective visual quality. In order to effectively apply these techniques, a method for effectively signaling relevant information is required. Summary of the invention

[0006] Technical Solution

[0007] According to an embodiment of this document, a method and device for improving image coding efficiency are provided.

[0008] According to an embodiment of this document, an effective filtering application method and device are provided.

[0009] According to the embodiments of this document, an effective LMCS application method and device are provided.

[0010] According to an embodiment of this document, a method and apparatus for hierarchically signaling ALF-related information are provided.

[0011] According to an embodiment of this document, a method and apparatus for hierarchically signaling LMCS-related information are provided.

[0012] According to an embodiment of this document, ALF data may be signaled through an APS, and APS ID information indicating an ID associated with a referenced APS may be signaled through header information (a picture header or a slice header).

[0013] According to an embodiment of this document, information about the number of APS IDs associated with ALF may be signaled through header information. In this case, the header information may include as many APS ID syntax elements as the number of APS IDs associated with ALF, and based on this, it may be possible to support deriving an adaptive filter (filter coefficient) in units of blocks / subblocks within the same picture or the same slice and applying ALF.

[0014] According to an embodiment of the present document, LMCS data may be signaled through an APS, and APS ID information indicating an ID of a referenced APS may be signaled through header information (a picture header or a slice header).

[0015] According to an embodiment of this document, a video / image decoding method performed by a decoding device is provided.

[0016] According to an embodiment of this document, there is provided a decoding device for performing video / image decoding.

[0017] According to an embodiment of this document, a video / image encoding method performed by an encoding device is provided.

[0018] According to an embodiment of this document, there is provided an encoding device for performing video / image encoding.

[0019] According to an embodiment of this document, a computer-readable digital storage medium is provided, which stores encoded video / image information generated according to the video / image encoding method disclosed in at least one embodiment of this document.

[0020] According to an embodiment of the present document, a computer-readable digital storage medium storing coding information or coded video / image information is provided, wherein the coding information or coded video / image information enables a decoding device to perform a video / image decoding method disclosed in at least one embodiment of the present document.

[0021] Effects of the Invention

[0022] According to the present disclosure, the overall image / video compression efficiency can be improved.

[0023] According to the present disclosure, subjective / objective visual quality can be improved through effective filtering.

[0024] According to the present disclosure, ALF and / or LMCS may be adaptively applied in units of pictures, slices, and / or coding blocks.

[0025] According to the present disclosure, information related to the ALF can be efficiently signaled.

[0026] According to the present disclosure, information related to LMCS can be efficiently signaled. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 An example of a video / image coding system to which embodiments of the present disclosure can be applied is schematically shown.

[0028] Figure 2 is a diagram schematically describing a configuration of a video / image encoding device to which an embodiment of the present disclosure can be applied.

[0029] Figure 3 is a diagram schematically describing a configuration of a video / image decoding device to which an embodiment of the present disclosure can be applied.

[0030] Figure 4 The hierarchical structure of the compiled image / video is shown exemplarily.

[0031] Figure 5 is a flow chart schematically illustrating an example of an ALF process.

[0032] Figure 6 An example of the shape of the ALF filter is shown.

[0033] Figure 7 An example of the hierarchical structure of ALF data is shown.

[0034] Figure 8 Another example of the hierarchical structure of ALF data is shown.

[0035] Fig. 9 The hierarchical structure of the CVS according to the embodiment of this document is exemplarily shown.

[0036] Fig.10 An exemplary LMCS structure according to an embodiment of this document is shown.

[0037] Fig.11 An exemplary LMCS structure according to another embodiment of this document is shown.

[0038] Fig.12 A diagram representing an exemplary forward map is shown.

[0039] Fig.13 and Fig.14 An example of a video / image encoding method and related components according to an embodiment of this document is schematically represented.

[0040] Fig.15 and Fig.16An example of a video / image decoding method and related components according to an embodiment of this document is schematically represented.

[0041] Fig.17 The content streaming system structure diagram to which the present disclosure is applied is illustratively shown. DETAILED DESCRIPTION

[0042] The disclosure of this document can be modified in various forms, and specific embodiments thereof will be described and illustrated in the accompanying drawings. However, the embodiments are not intended to limit the present disclosure. The terms used in this document are only used to describe specific embodiments, rather than to limit the embodiments of this document. Singular expressions include plural expressions as long as it is not clearly understood differently. Terms such as "including" and "having" are intended to indicate the presence of features, quantities, steps, operations, elements, components, or combinations thereof used in the document, and therefore it should be understood that the possibility of the presence or addition of one or more different features, quantities, steps, operations, elements, components, or combinations thereof is not excluded.

[0043] In addition, each configuration of the drawings described in this document is an independent diagram for explaining the functions of features that are different from each other, and does not mean that each configuration is implemented by different hardware or different software. For example, two or more configurations can be combined to form a configuration, and a configuration can also be divided into multiple configurations. Without departing from the gist of the present disclosure, embodiments of combined and / or separated configurations are included in the scope of the disclosure of this document.

[0044] Hereinafter, embodiments of the present document will be described in detail with reference to the accompanying drawings. In addition, in all drawings, the same reference numerals may be used to indicate the same elements, and the same description of the same elements will be omitted.

[0045] Figure 1 An example of a video / image coding system to which embodiments of this document can be applied is shown.

[0046] Reference Figure 1 The video / image coding system may include a source device and a receiving device. The source device may transmit the encoded video / image information or data in the form of a file or stream to the receiving device via a digital storage medium or a network.

[0047] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display, and the display may be configured as a separate device or an external component.

[0048] The video source may acquire the video / image by capturing, synthesizing or generating a video / image process. The video source may include a video / image capturing device, and / or a video / image generating device. For example, the video / image capturing device may include one or more cameras, a video / image archive including previously captured videos / images, etc. For example, the video / image generating device may include a computer, a tablet computer, and a smart phone, and may (electronically) generate the video / image. For example, a virtual video / image may be generated by a computer, etc. In this case, the video / image capturing process may be replaced by a process that generates relevant data.

[0049] The encoding device can encode the input video / image. For compression and coding efficiency, the encoding device can perform a series of processes such as prediction, transformation and quantization. The encoded data (encoded video / image information) can be output in the form of a bit stream.

[0050] The transmitter may transmit the encoded image / image information or data output in the form of a bit stream to a receiver of a receiving device in the form of a file or stream via a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter may include an element for generating a media file in a predetermined file format, and may include an element for transmitting via a broadcast / communication network. The receiver may receive / extract a bit stream and transmit the received bit stream to a decoding device.

[0051] The decoding device may decode a video / image by performing a series of processes corresponding to the operations of the encoding device, such as dequantization, inverse transformation, and prediction.

[0052] The renderer may render the decoded video / image. The rendered video / image may be displayed through a display.

[0053] This document relates to video / image coding. For example, the methods / embodiments disclosed in this document may be applied to methods disclosed in the Versatile Video Coding (VVC) standard. In addition, the methods / embodiments disclosed in this document may be applied to methods disclosed in the Basic Video Coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the 2nd generation Audio Video Coding standard (AVS2), or the next generation video / image coding standards (e.g., H.267, H.268, etc.).

[0054] The present disclosure proposes various embodiments of video / image coding, and unless otherwise specified, the above embodiments may also be performed in combination with each other.

[0055] In the present disclosure, video may refer to a series of images over time. A picture generally refers to a unit representing an image at a specific time frame, and a slice / tile refers to a unit that constitutes a part of a picture in terms of coding. A slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / tiles. A tile is a rectangular area of ​​a CTU within a specific tile column and a specific tile row in a picture (a tile is a rectangular area of ​​a CTU within a specific tile column and a specific tile row in a picture). A tile column is a rectangular area of ​​a CTU having a height equal to the height of the picture and a width that can be specified by a syntax element in a picture parameter set (a tile column is a rectangular area of ​​a CTU having a height equal to the height of the picture and a width specified by a syntax element in a picture parameter set). A tile row is a rectangular area of ​​a CTU having a width specified by a syntax element in a picture parameter set and a height that can be equal to the height of the picture (a tile row is a rectangular area of ​​a CTU having a height specified by a syntax element in a picture parameter set and a width equal to the width of the picture). Patch scan may refer to a specific sequential ordering of CTUs that partition a picture, and CTUs may be ordered consecutively in a CTU raster scan in a patch, and patches in a picture may be ordered consecutively in a raster scan of patches of a picture (patch scan is a specific sequential ordering of CTUs that partition a picture, where CTUs are ordered consecutively in a CTU raster scan in a patch and patches in a picture are ordered consecutively in a raster scan of patches of a picture). A slice includes an integer number of complete patches that may be exclusively contained in a single NAL unit or an integer number of consecutive complete CTU rows within a patch of a picture.

[0056] Meanwhile, a picture can be divided into two or more sub-pictures. A sub-picture can be a rectangular area of ​​one or more slices within a picture.

[0057] A pixel or a picture element (pel) may refer to the smallest unit constituting a picture (or image). In addition, a "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a value of a pixel, and may represent a pixel / pixel value of only a luminance component or a pixel / pixel value of only a chrominance component.

[0058] A unit may represent a basic unit of image processing. A unit may include at least one of a specific area of ​​a picture and information related to the area. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, a unit may be used interchangeably with terms such as a block or an area. In general, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients of M columns and N rows.

[0059] In this document, "A or B" may mean "only A", "only B", or "both A and B". In other words, "A or B" in this document may be interpreted as "A and / or B". For example, in this document, "A, B, or C" means "only A", "only B", "only C", or "any combination of A, B, and C".

[0060] As used in this document, a slash ( / ) or a comma (,) may mean "and / or". For example, "A / B" may mean "A and / or B". Thus, "A / B" may mean "only A", "only B", or "both A and B". For example, "A, B, C" may mean "A, B, or C".

[0061] In this document, "at least one of A and B" may mean "only A", "only B", or "both A and B". In addition, in this document, the expression "at least one of A or B" or "at least one of A and / or B" may be interpreted as the same as "at least one of A and B".

[0062] Furthermore, in this document, “at least one of A, B, and C” means “only A”, “only B”, “only C”, or “any combination of A, B, and C”. Furthermore, “at least one of A, B, or C” or “at least one of A, B and / or C” may mean “at least one of A, B, and C”.

[0063] In addition, brackets used in this document may mean "for example". Specifically, when "prediction (intra-frame prediction)" is indicated, "intra-frame prediction" may be proposed as an example of "prediction". In other words, "prediction" in this document is not limited to "intra-frame prediction", and "intra-frame prediction" may be proposed as an example of "prediction". In addition, even when "prediction (ie, intra-frame prediction)" is indicated, "intra-frame prediction" may be proposed as an example of "prediction".

[0064] Technical features described separately in one drawing in this document may be implemented separately or may be implemented simultaneously.

[0065] Figure 2 is a diagram schematically showing a configuration of a video / image encoding device to which an embodiment of the present disclosure can be applied. Hereinafter, a device referred to as an encoding device may include an image encoding device and / or a video encoding device.

[0066] Reference Figure 2, the encoding device 200 includes an image segmenter 210, a predictor 220, a residual processor 230 and an entropy encoder 240, an adder 250, a filter 260 and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234 and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. According to an embodiment, the image segmenter 210, the predictor 220, the residual processor 230, the entropy encoder 240, the adder 250 and the filter 260 may be configured by at least one hardware component (e.g., an encoder chipset or processor). In addition, the memory 270 may include a decoded picture buffer (DPB), or may be configured by a digital storage medium. The hardware component may also include the memory 270 as an internal / external component.

[0067] The image divider 210 may divide the input image (or picture, frame) input to the encoding device 200 into one or more processing units. As an example, the processing unit may be referred to as a coding unit (CU). In this case, the coding unit may be recursively divided from a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree binary tree ternary tree (QTBTTT) structure. For example, one coding unit may be divided into a plurality of coding units with a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. In this case, for example, the quadtree structure is first applied, and the binary tree structure and / or the ternary tree structure may be applied later. Alternatively, the binary tree structure may also be applied first. The coding process according to this document may be performed based on the final coding unit that is no longer divided. In this case, based on the coding efficiency according to image characteristics, etc., the maximum coding unit may be directly used as the final coding unit, or, as required, the coding unit may be recursively divided into coding units with a deeper depth, so that the coding unit with the best size may be used as the final coding unit. Here, the coding process may include processes such as prediction, transformation, and reconstruction described later. As another example, the processing unit may also include a prediction unit (PU) or a transform unit (TU). In this case, each of the prediction unit and the transform unit may be split or partitioned from the above-mentioned final coding unit. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from a transform coefficient.

[0068] In some cases, a unit may be used interchangeably with terms such as a block or region. In general, an M×N block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component. A sample may be used as a term corresponding to one picture (or image) of a pixel or a pixel element.

[0069] In the encoding device 200, a prediction signal (prediction block, prediction sample array) output from the inter predictor 221 or the intra predictor 222 is subtracted from an input image signal (original block, original sample array) to generate a residual signal (residual block, residual sample array), and the generated residual signal is sent to the transformer 232. In this case, as shown, a unit that subtracts a prediction signal (prediction block, prediction sample array) from an input image signal (original block, original sample array) in the encoder 200 may be referred to as a subtractor 231. The predictor may perform prediction on a block to be processed (hereinafter, referred to as a current block) and generate a prediction block including prediction samples of the current block. The predictor may determine whether intra prediction or inter prediction is applied based on the current block or CU. As described later in the description of each prediction mode, the predictor may generate various types of information related to prediction (e.g., prediction mode information) and send the generated information to the entropy encoder 240. Information about the prediction may be encoded in the entropy encoder 240 and output in the form of a bitstream.

[0070] The intra-frame predictor 222 may predict the current block with reference to samples in the current picture. Depending on the prediction mode, the referenced samples may be located near the current block or may be spaced apart. In intra-frame prediction, the prediction mode may include multiple non-directional modes and multiple directional modes. For example, the non-directional mode may include a DC mode and a plane mode. For example, depending on the level of detail of the prediction direction, the directional mode may include 33 directional prediction modes or 65 directional prediction modes. However, this is only an example, and more or fewer directional prediction modes may be used depending on the settings. The intra-frame predictor 222 may use the prediction mode applied to the neighboring blocks to determine the prediction mode applied to the current block.

[0071] The inter-frame predictor 221 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. Here, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter-frame predictor 221 may configure a motion information candidate list based on the neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter-frame prediction may be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter-frame predictor 221 may use the motion information of the neighboring block as the motion information of the current block. In skip mode, unlike merge mode, a residual signal may not be transmitted. The motion vector prediction (MVP) mode may indicate the motion vector of the current block by using the motion vector of the neighboring block as a motion vector predictor and signaling the motion vector difference.

[0072] The predictor 200 may generate a prediction signal based on various prediction methods described later. For example, the predictor may not only apply intra prediction or inter prediction to predict a block, but may also apply both intra prediction and inter prediction at the same time. This may be referred to as combined inter and intra prediction (CIIP). In addition, the predictor may be based on an intra block copy (IBC) prediction mode or a palette mode to perform prediction on a block. The IBC prediction mode or the palette mode may be used for content image / video coding of games, etc., such as screen content coding (SCC). IBC basically performs prediction in the current picture, but it may be performed similarly to inter prediction because it derives a reference block in the current picture. That is, IBC may use at least one of the inter prediction techniques described in this document. The palette mode may be considered an example of intra coding or intra prediction. When the palette mode is applied, the sample values ​​in the picture may be signaled based on information about the palette table and the palette index.

[0073] The prediction signal generated by the predictor (including the inter-frame predictor 221 and / or the intra-frame predictor 222) can be used to generate a reconstruction signal or to generate a residual signal. The transformer 232 can generate a transform coefficient by applying a transform technique to the residual signal. For example, the transform technique may include at least one of the following: discrete cosine transform (DCT), discrete sine transform (DST), graph-based transform (GBT), or conditional nonlinear transform (CNT). Here, when the relationship information between pixels is illustrated as a graph, GBT means a transform obtained from a graph. CNT means a transform obtained based on a prediction signal generated by using all previously reconstructed pixels. In addition, the transform process can also be applied to pixel blocks of squares of the same size, and can also be applied to blocks of variable sizes other than squares.

[0074] The quantizer 233 may quantize the transform coefficients to send the quantized transform coefficients to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information about the quantized transform coefficients) into a bitstream. Information about the quantized transform coefficients may be referred to as residual information. The quantizer 233 may rearrange the quantized transform coefficients in a block form in a one-dimensional vector form based on the coefficient scanning order, and also generate information about the quantized transform coefficients based on the quantized transform coefficients in a one-dimensional vector form. The entropy encoder 240 may perform various encoding methods such as exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), etc. The entropy encoder 240 may also encode information necessary for reconstructing the video / image (e.g., the value of a syntax element, etc.) in addition to the quantized transform coefficients together or separately. The encoded information (e.g., encoded video / image information) may be transmitted or stored in units of a network abstraction layer (NAL) in the form of a bitstream. The video / image information may also include information about various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. The information and / or syntax elements that are signaled / sent later described in this disclosure may be encoded by the aforementioned encoding process and thus included in the bitstream. The bitstream may be sent over a network or may be stored in a digital storage medium. Here, the network may include a broadcast network and / or a communication network, etc., and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) for sending a signal output from the entropy encoder 240 and / or a storage unit (not shown) for storing the signal may be configured as an internal / external element of the encoding device 200, or the transmitter may also be included in the entropy encoder 240.

[0075] The quantized transform coefficients output from the quantizer 233 can be used to generate a prediction signal. For example, the dequantizer 234 and the inverse transformer 235 apply dequantization and inverse transformation to the quantized transform coefficients so that the residual signal (residual block or residual sample) can be reconstructed. The adder 250 adds the reconstructed residual signal to the prediction signal output from the inter-frame predictor 221 or the intra-frame predictor 222, so that a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) can be generated. If there is no residual in the block to be processed, the prediction block can be used as a reconstructed block, such as in the case of applying the skip mode. The adder 250 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture, and as described later, it is also used for inter-frame prediction of the next picture through filtering.

[0076] Meanwhile, luminance mapping and chrominance scaling (LMCS) may also be applied during picture encoding and / or reconstruction.

[0077] The filter 260 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 260 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and store the modified reconstructed picture in the memory 270 (specifically, the DPB of the memory 270). For example, various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filter 260 can generate various types of information related to filtering and send the generated information to the entropy encoder 240, as described later in the description of each filtering method. The information related to filtering can be encoded by the entropy encoder 240 and output in the form of a bitstream.

[0078] The modified reconstructed picture transmitted to the memory 270 may be used as a reference picture in the inter predictor 221. When inter prediction is applied by the encoding apparatus, prediction mismatch between the encoding apparatus 200 and the decoding apparatus may be avoided and coding efficiency may be improved.

[0079] The DPB of the memory 270 DPB may store a modified reconstructed picture used as a reference picture in the inter-frame predictor 221. The memory 270 may store motion information of a block from which motion information in the current picture is derived (or encoded) and / or motion information of a reconstructed block in the picture. The stored motion information may be sent to the inter-frame predictor 221 and used as motion information of a spatially adjacent block or motion information of a temporally adjacent block. The memory 270 may store reconstructed samples of a reconstructed block in the current picture and may transmit the reconstructed samples to the intra-frame predictor 222.

[0080] Figure 3is a diagram for schematically explaining a configuration of a video / image decoding device to which an embodiment of the present disclosure can be applied. Hereinafter, a device referred to as a decoding device may include an image decoding device and / or a video decoding device.

[0081] Reference Figure 3 , the decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 331 and an intra-frame predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 321. According to an embodiment, the entropy decoding 310, the residual processor 320, the predictor 330, the adder 340, and the filter 350 may be configured by a hardware component (e.g., a decoder chipset or a processor). In addition, the memory 360 may include a decoded picture buffer (DPB) or may be configured by a digital storage medium. The hardware component may also include the memory 360 as an internal / external component.

[0082] When a bit stream including video / image information is input, the decoding device 300 may respond to the Figure 2 The image can be reconstructed by processing video / image information in the encoding device shown. For example, the decoding device 300 can derive the unit / block based on the block segmentation related information obtained from the bit stream. The decoding device 300 can perform decoding using a processing unit applied to the encoding device. Therefore, the processing unit for decoding can be, for example, a coding unit, and the coding unit can be divided from the coding tree unit or the maximum coding unit according to a quadtree structure, a binary tree structure and / or a ternary tree structure. One or more transform units can be derived from the coding unit. In addition, the reconstructed image signal decoded and output by the decoding device 300 can be reproduced by a reproduction device.

[0083] The decoding device 300 may receive the bit stream from Figure 2, and the received signal can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can derive the information (e.g., video / image information) required for image reconstruction (or picture reconstruction) by parsing the bitstream. The video / image information may also include information about various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. The decoding device may also decode the picture based on the information about the parameter set and / or the general constraint information. The signaled / received information and / or syntax elements described later in the present disclosure may be decoded by a decoding process and obtained from the bitstream. For example, the entropy decoder 310 may decode the information within the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the residual-related transform coefficients. More specifically, the CABAC entropy decoding method may receive a bin (binary bit) corresponding to each syntax element from a bitstream, determine a context model using information of syntax elements to be decoded, information of decoded adjacent blocks or blocks to be decoded, or information of symbols / bins decoded in a previous stage, and generate symbols corresponding to the values ​​of each syntax element by predicting the probability of generating bins according to the determined context model to perform arithmetic decoding on bins. At this point, the CABAC entropy decoding method may determine a context model and then update the context model by using the information of the decoded symbol / bin for the context model of the next symbol / bin. Information related to prediction among the information decoded by the entropy decoder 310 may be provided to a predictor (inter-frame predictor 332 and intra-frame predictor 331), and residual values ​​(i.e., quantized transform coefficients and related parameter information) entropy decoded by the entropy decoder 310 may be input to a residual processor 320. The residual processor 320 may derive a residual signal (residual block, residual sample, residual sample array). In addition, information about filtering among the information decoded by the entropy decoder 310 may be provided to a filter 350. Meanwhile, a receiver (not shown) for receiving a signal output from the encoding device may also be configured as an internal / external element of the decoding device 300, or the receiver may be a component of the entropy decoder 310. Meanwhile, the decoding device according to the present disclosure may be referred to as a video / image / picture decoding device, and the decoding device may be classified into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of the following: a dequantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.

[0084] The dequantizer 321 may dequantize the quantized transform coefficient and output the transform coefficient. The dequantizer 321 may rearrange the quantized transform coefficient in a two-dimensional block form. In this case, the rearrangement may be performed based on the coefficient scanning order performed in the encoding device. The dequantizer 321 may perform dequantization on the quantized transform coefficient using a quantization parameter (e.g., quantization step size information) and obtain the transform coefficient.

[0085] The inverse transformer 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0086] The predictor 330 may perform prediction on the current block and generate a prediction block including prediction samples of the current block. The predictor may determine whether to apply intra prediction or inter prediction to the current block based on information on prediction output from the entropy decoder 310 and may determine a specific intra / inter prediction mode.

[0087] The predictor may generate a prediction signal based on various prediction methods described below. For example, the predictor may not only apply intra prediction or inter prediction to predict a block, but may also apply intra prediction and inter prediction at the same time. This may be referred to as combined inter and intra prediction (CIIP). In addition, the predictor may predict a block based on an intra block copy (IBC) prediction mode or a palette mode. The IBC prediction mode or the palette mode may be used for content image / video coding of games, such as screen content coding (SCC). IBC basically performs a prediction in the current picture, but may be performed similarly to inter prediction so that a reference block is derived in the current picture. That is, IBC may use at least one inter prediction technique described herein. The palette mode may be considered as an example of intra coding or intra prediction. When the palette mode is applied, information about the palette table and the palette index may be included in the video / image information and signaled.

[0088] The intra-frame predictor 331 may predict the current block with reference to samples in the current picture. Depending on the prediction mode, the referenced samples may be located near the current block or may be spaced apart. In intra-frame prediction, the prediction mode may include a plurality of non-directional modes and a plurality of directional modes. The intra-frame predictor 331 may use the prediction mode applied to the neighboring blocks to determine the prediction mode applied to the current block.

[0089] The inter-frame predictor 332 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter-frame predictor 332 may configure a motion information candidate list based on the neighboring blocks and derive a motion vector and / or a reference picture index of the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes, and information about the prediction may include information indicating the inter-frame prediction mode of the current block.

[0090] The adder 340 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including the inter-frame predictor 332 and / or the intra-frame predictor 331). If there is no residual in the block to be processed, for example, when the skip mode is applied, the prediction block can be used as the reconstructed block.

[0091] The adder 340 may be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next block to be processed in the current picture, may be output through filtering as described below, or may be used for inter prediction of the next picture.

[0092] Meanwhile, luminance mapping and chrominance scaling (LMCS) may be applied in the picture decoding process.

[0093] The filter 350 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 350 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and store the modified reconstructed picture in the memory 360 (specifically, the DPB of the memory 360). For example, the various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0094] The (modified) reconstructed picture stored in the DPB of the memory 360 may be used as a reference picture in the inter-frame predictor 332. The memory 360 may store the motion information of the block in which the motion information in the current picture is derived (decoded) and / or the motion information of the block in the previously reconstructed picture. The stored motion information may be transmitted to the inter-frame predictor 260 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 360 may store the reconstructed samples of the reconstructed block in the current picture and transmit the stored reconstructed samples to the intra-frame predictor 331.

[0095] In this document, the exemplary embodiments described in the filter 260, the inter-frame predictor 221, and the intra-frame predictor 222 of the encoding device 200 may be equally applied to or correspond to the filter 350, the inter-frame predictor 332, and the intra-frame predictor 331 of the decoding device 300, respectively.

[0096] Meanwhile, as described above, when performing video coding, prediction is performed to improve compression efficiency. Thus, a prediction block including prediction samples for a current block as a block to be coded (i.e., a coding target block) may be generated. Here, the prediction block includes prediction samples in a spatial domain (or a pixel domain). The prediction block is derived in the same manner in the encoding device and the decoding device, and the encoding device may signal information about the residual between the original block and the prediction block (residual information) instead of the original sample value of the original block to the decoding device, thereby improving image coding efficiency. The decoding device may derive a residual block including residual samples based on the residual information, add the residual block and the prediction block to generate a reconstructed block including reconstructed samples, and generate a reconstructed picture including the reconstructed block.

[0097] The residual information may be generated by a transformation and quantization procedure. For example, the encoding device may derive a residual block between the original block and the prediction block, perform a transformation process on the residual samples (residual sample array) included in the residual block to derive a transformation coefficient, perform a quantization process on the transformation coefficient to derive a quantized transformation coefficient, and notify the decoding device of the relevant residual information (through a bitstream) with a signal. Here, the residual information may include value information, position information, transformation technology, transformation kernel, quantization parameter, etc. of the quantized transformation coefficient. The decoding device may perform a dequantization / inverse transformation process based on the residual information and derive residual samples (or residual blocks). The decoding device may generate a reconstructed picture based on the prediction block and the residual block. In addition, for reference for inter-frame prediction of subsequent pictures, the encoding device may also dequantize / inverse transform the quantized transformation coefficient to derive a residual block, and generate a reconstructed picture based on the residual block.

[0098] Intra-frame prediction may indicate a prediction of a prediction sample for a current block generated based on a reference sample in a picture to which the current block belongs (hereinafter referred to as the current picture). When intra-frame prediction is applied to the current block, neighboring reference samples to be used for intra-frame prediction of the current block may be derived. The neighboring reference samples of the current block may include: samples adjacent to the left boundary and a total of 2xnH samples adjacent to the lower left of the current block of size nWxnH; samples adjacent to the upper boundary of the current block and a total of 2xnW samples adjacent to the upper right of the current block; and one sample adjacent to the upper left of the current block. Alternatively, the neighboring reference samples of the current block may include multiple columns of upper neighboring samples and multiple rows of left neighboring samples. In addition, the neighboring reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block of size nWxnH, a total of nW samples adjacent to the lower boundary of the current block, and one sample adjacent to the lower right of the current block.

[0099] However, some of the neighboring reference samples of the current block have not been decoded or may not be available. In this case, the decoding device can construct the neighboring reference samples to be used for prediction by replacing the unavailable samples with available samples. Alternatively, the neighboring reference samples to be used for prediction can be constructed by interpolation of available samples.

[0100] When deriving the neighboring reference samples, (i) the prediction sample may be derived based on an average or interpolation of the neighboring reference samples of the current block, or (ii) the prediction sample may be derived based on a reference sample located in a specific (prediction) direction relative to the prediction sample among the neighboring reference samples of the current block. The case of (i) may be referred to as a non-directional mode or a non-angular mode, and the case of (ii) may be referred to as a directional mode or an angular mode.

[0101] In addition, the prediction sample can be generated by interpolating the first neighboring sample located in the prediction direction of the intra prediction mode of the current block based on the prediction sample of the current block among the adjacent reference samples and the second neighboring sample located in the direction opposite to the prediction direction. The above situation can be called linear interpolation intra prediction (LIP). In addition, the chrominance prediction sample can be generated based on the luma sample using a linear model (LM). This situation can be called LM mode.

[0102] In addition, the temporary prediction sample of the current block can be derived by the neighboring reference samples based on the filtering, and the prediction sample of the current block can also be derived by the weighted sum of at least one reference sample derived according to the intra prediction mode among the existing neighboring reference samples (i.e., the unfiltered neighboring reference samples) and the temporary prediction sample. The above situation can be called position-dependent intra prediction (PDPC).

[0103] In addition, a reference sample line with the highest prediction accuracy is selected among multiple reference sample lines surrounding the current block, and a reference sample located in a prediction direction of the selected reference sample line is used to derive a prediction sample, and intra-frame prediction encoding can be performed by indicating (notifying with a signal) the used reference sample line to a decoding device. The above situation may be referred to as multi-reference line intra-frame prediction or MRL-based intra-frame prediction.

[0104] In addition, the current block is divided into vertical or horizontal sub-partitions to perform intra prediction based on the same intra prediction mode, but it is possible to derive and use adjacent reference samples in units of sub-partitions. That is, in this case, the intra prediction mode for the current block can be applied to all sub-partitions in the same manner, but the intra prediction performance can be improved by deriving and using adjacent reference samples in units of sub-partitions in some cases. This prediction method may be referred to as intra prediction based on intra sub-partitions (ISP).

[0105] The above-mentioned intra-frame prediction method may be referred to as an intra-frame prediction type to distinguish it from an intra-frame prediction mode. The intra-frame prediction type may be referred to by various terms such as an intra-frame prediction technique or an additional intra-frame prediction mode. For example, the intra-frame prediction type (or additional intra-frame prediction mode) may include at least one of the LIP, PDPC, MRL and ISP mentioned above. General intra-frame prediction methods other than specific intra-frame prediction types such as LIP, PDPC, MRL and ISP may be referred to as normal intra-frame prediction types. Normal intra-frame prediction types may generally be applied when a specific intra-frame prediction type is not applied, and prediction may be performed based on an intra-frame prediction mode. At the same time, if necessary, post-processing filtering may be performed on the derived prediction samples.

[0106] Specifically, the intra prediction process may include an intra prediction mode / type determination step, an adjacent reference sample derivation step, and a prediction sample derivation step based on the intra prediction mode / type. In addition, if necessary, a post-filtering step may be performed on the derived prediction samples.

[0107] When intra prediction is applied, the intra prediction mode applied to the current block can be determined by using the intra prediction mode of the neighboring block. For example, the decoding device can select one of the PMP candidates in the MPM list derived based on the intra prediction mode of the neighboring block (e.g., left and / or upper neighboring block) of the current block and the additional candidate mode based on the received MPM (most probable mode) index, or can select one of the remaining intra prediction modes not included in the MPM candidate (and plane mode) based on the remaining intra prediction mode information. The MPM list can be configured to include or exclude the plane mode as a candidate. For example, when the MPM list includes the plane mode as a candidate, the MPM list may have 6 candidates, and when the MPM list does not include the plane mode as a candidate, the MPM list may have 5 candidates. When the MPM list does not include the plane mode as a candidate, a non-planar flag (e.g., intra_luma_not_planar_flag) indicating whether the intra prediction mode of the current block is not a plane mode can be signaled. For example, the MPM flag can be first signaled, and the MPM index and the non-planar flag can be signaled when the value of the MPM flag is 1. In addition, the MPM index may be signaled when the value of the non-planar flag is 1. Here, the reason why the MPM list is configured to not include the planar mode as a candidate is because the planar mode is always considered to be an MPM and therefore the planar mode is first confirmed by first signaling the flag (non-planar flag) rather than saying that the planar mode is not an MPM.

[0108] For example, it may be indicated based on an MPM flag (e.g., intra_luma_mpm_flag) whether the intra prediction mode applied to the current block is in the MPM candidate (and planar mode) or in the remaining mode. A value of 1 for the MPM flag may indicate that the intra prediction mode for the current block is within the MPM candidate (and planar mode), and a value of 0 for the MPM flag may indicate that the intra prediction mode for the current block is not within the MPM candidate (and planar mode). A non-planar flag (e.g., intra_luma_not_planar_flag) value of 0 may indicate that the intra prediction mode for the current block is a planar mode, and a non-planar flag value of 1 may indicate that the intra prediction mode for the current block is not a planar mode. The MPM index may be signaled in the form of an mpm_idx or intra_luma_mpm_idx syntax element, and the remaining intra prediction mode information may be signaled in the form of a rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax element. For example, the remaining intra prediction mode information may indicate one of the remaining intra prediction modes that is not included in the MPM candidates (and planar mode) among all intra prediction modes by indexing in the order of the prediction mode number. The intra prediction mode may be an intra prediction mode for a luma component (sample). In the following, the intra prediction mode information may include an MPM flag (e.g., intra_luma_mpm_flag), a non-planar flag (e.g., intra_luma_not_planar_flag), an MPM index (e.g., mpm_idx or intra_luma_mpm_idx), and remaining intra prediction mode information (rem_intra_luma_pred_mode or intra_luma_mpm_remainder). In this document, the MPM list may be referred to by various terms such as MPM candidate list and candModeList. When a MIP is applied to the current block, a separate mpm flag (eg, intra_mip_mpm_flag), an mpm index (eg, intra_mip_mpm_idx), and remaining intra prediction mode information (eg, intra_mip_mpm_remaining) for the MIP may be separately signaled, and the non-planar flag may not be signaled.

[0109] In other words, generally, when an image is divided into blocks, the current block to be coded and the neighboring blocks have similar image characteristics. Therefore, the possibility that the current block and the neighboring blocks have the same or similar intra-frame prediction mode is high. Therefore, the encoding device can use the intra-frame prediction mode of the neighboring block to encode the intra-frame prediction mode of the current block.

[0110] For example, the encoding device / decoding device may configure a list of most probable modes (MPMs) for the current block. The MPM list may be referred to as an MPM candidate list. Here, MPM may refer to a mode for improving coding efficiency by considering the similarity between the current block and the neighboring blocks during intra-prediction mode coding. As described above, the MPM list may be configured to include a planar mode, or may be configured to exclude a planar mode. For example, when the MPM list includes a planar mode, the number of candidates in the MPM list may be six. And, when the MPM list does not include a planar mode, the number of candidates in the MPM list may be five.

[0111] The encoding device / decoding device can be configured with an MPM list including 5 or 6 MPMs.

[0112] To configure the MPM list, three types of modes may be considered: default intra mode, adjacent intra mode, and derived intra mode.

[0113] For adjacent intra mode, two neighboring blocks may be considered, ie, a left neighboring block and an upper neighboring block.

[0114] As described above, if the MPM list is configured not to include the planar mode, the planar mode is excluded from the list and the number of MPM list candidates may be set to five.

[0115] In addition, in the intra prediction mode, the non-directional mode (or non-angular mode) is a DC mode based on an average value of adjacent reference samples of the current block or a planar mode based on interpolation of adjacent reference samples.

[0116] When inter-frame prediction is applied, the predictor of the encoding device / decoding device can derive prediction samples by performing inter-frame prediction in units of blocks. Inter-frame prediction can be a prediction derived in a manner that depends on data elements (e.g., sample values ​​or motion information) of pictures other than the current picture. When inter-frame prediction is applied to the current block, the prediction block (prediction sample array) of the current block can be derived based on the reference block (reference sample array) specified by the motion vector on the reference picture indicated by the reference picture index. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information of the current block can be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include motion vectors and reference picture indexes. The motion information may also include inter-frame prediction type (L0 prediction, L1 prediction, Bi prediction, etc.) information. When inter-frame prediction is applied, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. A temporally neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc., and a reference picture including a temporally neighboring block may be referred to as a collocated picture (colPic). For example, a motion information candidate list may be constructed based on the neighboring blocks of the current block, and a flag or index information indicating which candidate is selected (used) to derive the motion vector and / or reference picture index of the current block may be signaled. Inter-frame prediction may be performed based on various prediction modes. For example, in skip mode and merge mode, the motion information of the current block may be the same as the motion information of the selected neighboring block. In skip mode, unlike merge mode, a residual signal may not be sent. In the case of a motion vector prediction (MVP) mode, the motion vector of the selected neighboring block may be used as a motion vector predictor, and the motion vector difference may be signaled. In this case, the motion vector of the current block may be derived using the sum of the motion vector predictor and the motion vector difference.

[0117] The motion information may include L0 motion information and / or L1 motion information according to the inter prediction type (L0 prediction, L1 prediction, Bi prediction, etc.). The motion vector in the L0 direction may be referred to as an L0 motion vector or MVL0, and the motion vector in the L1 direction may be referred to as an L1 motion vector or MVL1. Prediction based on the L0 motion vector may be referred to as L0 prediction, prediction based on the L1 motion vector may be referred to as L1 prediction, and prediction based on both the L0 motion vector and the L1 motion vector may be referred to as bidirectional prediction. Here, the L0 motion vector may indicate a motion vector associated with the reference picture list L0 (L0), and the L1 motion vector may indicate a motion vector associated with the reference picture list L1 (L1). Reference picture list

[0118] L0 may include a picture that precedes the current picture in output order as a reference picture, and the reference picture

[0119] The reference picture list L1 may include pictures that are after the current picture in the output order. The previous picture may be referred to as a forward (reference) picture, and the subsequent picture may be referred to as a backward (reference) picture. The reference picture list L0 may also include pictures that are after the current picture in the output order as reference pictures. In this case, in the reference picture list L0, the previous picture may be indexed first and the subsequent picture may be indexed next. The reference picture list L1 may also include pictures that are before the current picture in the output order as reference pictures. In this case, the subsequent picture may be indexed first in the reference picture list 1, and the previous picture may be indexed next. Here, the output order may correspond to a picture order count (POC) order.

[0120] Figure 4 The hierarchical structure of the compiled image / video is shown exemplarily.

[0121] refer to Figure 4 , the encoded image / video is divided into VCL (Video Coding Layer) which handles the image / video decoding process and itself, a subsystem that sends and stores the coding information, and a Network Abstraction Layer (NAL) which exists between the VCL and the subsystem and is responsible for the network adaptation function.

[0122] VCL can generate VCL data including compressed image data (slice data), or generate parameter sets including picture parameter sets (Picture Parameter Set: PPS), sequence parameter sets (Sequence Parameter Set: SPS), video parameter sets (Video Parameter Set: VPS), etc., or supplementary enhancement information (SEI) messages that are additionally required for the decoding process of the image.

[0123] In NAL, a NAL unit can be generated by adding header information (NAL unit header) to a raw byte sequence payload (RBSP) generated in the VCL. In this case, the RBSP refers to slice data, parameter sets, SEI messages, etc. generated in the VCL. The NAL unit header may include NAL unit type information specified according to the RBSP data included in the corresponding NAL unit.

[0124] As shown in the figure, the NAL unit can be divided into a VCL NAL unit and a non-VCL NAL unit according to the RBSP generated in the VCL. The VCL NAL unit may mean a NAL unit including information about the image (slice data), and the non-VCL NAL unit may mean a NAL unit containing information required for decoding the image (parameter set or SEI message).

[0125] The above-mentioned VCL NAL unit and non-VCL NAL unit can be sent through the network by adding header information according to the data standard of the subsystem. For example, the NAL unit can be converted into a data form of a predetermined standard, such as H.266 / VVC file format, real-time transport protocol (RTP), transport stream (TS), etc., and sent through various networks.

[0126] As described above, in a NAL unit, a NAL unit type may be specified according to an RBSP data structure included in a corresponding NAL unit, and information on the NAL unit type may be stored in a NAL unit header and signaled in the NAL unit header.

[0127] For example, NAL units may be roughly classified into VCL NAL unit types and non-VCL NAL unit types depending on whether the NAL unit includes information about an image (slice data). VCL NAL unit types may be classified according to the attributes and types of pictures included in the VCL NAL unit, and non-VCL NAL unit types may be classified according to the types of parameter sets.

[0128] The following are examples of NAL unit types specified according to the type of parameter sets included in a non-VCL NAL unit type.

[0129] -APS (Adaptation Parameter Set) NAL unit: the type of NAL unit that includes APS

[0130] -DPS (Decoding Parameter Set) NAL unit: the type of NAL unit that includes the DPS

[0131] -VPS (Video Parameter Set) NAL unit: the type of NAL unit that contains the VPS

[0132] -SPS (Sequence Parameter Set) NAL unit: the type of NAL unit that includes SPS

[0133] -PPS (Picture Parameter Set) NAL unit: the type of NAL unit that includes PPS

[0134] - PH (Picture Header) NAL unit: the type of NAL unit including PH

[0135] The above-mentioned NAL unit type has syntax information of the NAL unit type, and the syntax information may be stored and signaled in the NAL unit header. For example, the syntax information may be nal_unit_type, and the NAL unit type may be specified by the nal_unit_type value.

[0136] In addition, as described above, a picture may include multiple slices, and a slice may include a slice header and slice data. In this case, a picture header may also be added to multiple slices (slice header and slice data set) in a picture. The picture header (picture header syntax) may include information / parameters generally applicable to pictures. In this document, slices may be mixed with or replaced with tiling groups. In addition, in this document, slice headers may be mixed with or replaced with tiling group headers.

[0137] A slice header (slice header syntax) may include information / parameters that are generally applicable to slices. APS (APS syntax) or PPS (PPS syntax) may include information / parameters that are generally applicable to one or more slices or pictures. SPS (SPS syntax) may include information / parameters that are generally applicable to one or more sequences. VPS (VPS syntax) may include information / parameters that are generally applicable to multiple layers. DPS (DPS syntax) may include information / parameters that are generally applicable to the entire video. DPS may include information / parameters related to the concatenation of a coded video sequence (CVS). In this document, a high-level syntax (HLS) may include at least one of the following: APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, picture header syntax, and slice header syntax.

[0138] In this document, the image / video information encoded in the encoding device and notified to the decoding device in the form of a bitstream may include, in addition to the picture segmentation related information in the picture, intra / inter prediction information, residual information, in-loop filtering information, etc., information included in the slice header, information included in the picture header, information included in the APS, information included in the PPS, information included in the SPS, information included in the VPS and / or information included in the DPS. In addition, the image / video information may also include information of the NAL unit header.

[0139] At the same time, in order to compensate for the difference between the original image and the reconstructed image due to errors occurring in the compression coding process such as quantization, an in-loop filtering process may be performed on the reconstructed sample or the reconstructed picture as described above. As described above, the in-loop filtering may be performed by a filter of an encoding device and a filter of a decoding device, and a deblocking filter, SAO and / or an adaptive loop filter (ALF) may be applied. For example, the ALF process may be performed after the deblocking filtering process and / or the SAO process are completed. However, even in this case, the deblocking filtering process and / or the SAO process may be omitted.

[0140] Figure 5 is a flow chart schematically illustrating an example of an ALF process. Figure 5The ALF process disclosed in can be executed in a coding device and a decoding device. In this document, a coding device may include a coding device and / or a decoding device.

[0141] refer to Figure 5 , the coding device derives a filter for ALF (S500). The filter may include a filter coefficient. The coding device may determine whether to apply ALF, and when it is determined to apply ALF, a filter including a filter coefficient for ALF may be derived. The filter (coefficient) for ALF or information about the derivation of the filter (coefficient) for ALF may be referred to as ALF parameters. Information about whether ALF is applied (e.g., an ALF availability flag) and ALF data for deriving the filter may be signaled from the coding device to the decoding device. The ALF data may include information for deriving the filter for ALF. In addition, for example, for hierarchical control of ALF, the ALF availability flag may be signaled at the SPS, picture header, slice header, and / or CTB level, respectively.

[0142] In order to derive the filter for ALF, the activity and / or directionality of the current block (or ALF target block) is derived, and the filter can be derived based on the activity and / or directionality. For example, the ALF process can be applied in units of 4×4 blocks (based on the luminance component). The current block or ALF target block can be, for example, a CU, or can be a 4x4 block within a CU. Specifically, for example, the filter for ALF can be derived based on a first filter derived from information included in the ALF data and a predefined second filter, and the coding device can select one of the filters based on the activity and / or directionality. The coding device can use the filter coefficients included in the selected filter for ALF.

[0143] The coding device performs filtering based on the filter (S510). Modified reconstructed samples can be derived based on filtering. For example, filter coefficients in the filter can be arranged or assigned according to the filter shape, and filtering can be performed on the reconstructed samples in the current block. Here, the reconstructed samples in the current block can be reconstructed samples after the deblocking filter process and the SAO process are completed. For example, one filter shape can be used, or a filter shape selected from a plurality of predetermined filter shapes can be used. For example, the filter shape applied to the luminance component and the filter shape applied to the chrominance component can be different. For example, a 7×7 diamond filter shape can be used for the luminance component, and a 5×5 diamond filter shape can be used for the chrominance component.

[0144] Figure 6 An example of the shape of the ALF filter is shown.

[0145] Figure 6(a) shows the shape of a 7×7 diamond filter, Figure 6 (b) shows the shape of a 5×5 diamond filter. Figure 6 In the filter shape, Cn represents the filter coefficient. When n in Cn is the same, this indicates that the same filter coefficient can be assigned. In this document, the position and / or unit of the filter coefficient assigned according to the filter shape of the ALF can be referred to as a filter tap. In this case, one filter coefficient can be assigned to each filter tap, and the arrangement of the filter taps can correspond to the filter shape. The filter tap located at the center of the filter shape can be called a center filter tap. The same filter coefficient can be assigned to two filter taps of the same n value, which exist at positions corresponding to each other relative to the center filter tap. For example, in the case of a 7×7 diamond filter shape, 25 filter taps are included, and since the filter coefficients C0 to C11 are assigned in a centrally symmetrical form, the filter coefficients can be assigned to the 25 filter taps using only 13 filter coefficients. In addition, for example, in the case of a 5×5 diamond filter shape, 13 filter taps are included, and since the filter coefficients C0 to C5 are assigned in a centrally symmetrical form, the filter coefficients can be assigned to the 13 filter taps using only 7 filter coefficients. For example, in order to reduce the amount of data of information about the signaled filter coefficients, 12 of the 13 filter coefficients of a 7×7 diamond filter shape are (explicitly) signaled, and 1 filter coefficient can be (implicitly) derived. In addition, for example, 6 of the 7 filter coefficients of a 5×5 diamond filter shape can be (explicitly) signaled, and 1 filter coefficient can be (implicitly) derived.

[0146] According to an embodiment of this document, ALF parameters for an ALF process may be signaled through an adaptation parameter set (APS). The ALF parameters may be derived from filter information for ALF or ALF data.

[0147] ALF is an in-loop filtering technique that can be applied to image / video coding as described above. ALF can be performed using an adaptive filter based on Wiener. This can minimize the mean square error (MSE) between the original sample and the decoded sample (or reconstructed sample). The advanced design of the ALF tool can be incorporated into the syntax elements accessible in the SPS and / or slice header (or mosaic group header).

[0148] Figure 7 An example of the hierarchical structure of ALF data is shown.

[0149] refer to Figure 7, a coded video sequence (CVS) may include an SPS, one or more PPSs, and subsequently one or more coded pictures. Each coded picture may be divided into a rectangular area. A rectangular area may be referred to as a mosaic. One or more mosaics may be aggregated to form a mosaic group or a slice. In this case, a mosaic group header may be linked to a PPS, and a PPS may be linked to an SPS. According to the prior art, ALF data (ALF parameters) are included in a mosaic group header. Considering that a video is composed of multiple pictures and a picture includes multiple mosaics, frequent ALF data (ALF parameters) signaling in units of mosaic groups reduces coding efficiency.

[0150] According to the embodiments proposed in this document, ALF parameters may be signaled by including in the APS as follows.

[0151] Figure 8 Another example of the hierarchical structure of ALF data is shown.

[0152] refer to Figure 8 , defines the APS, and the APS may carry necessary ALF data (ALF parameters). In addition, the APS may have self-identification parameters and ALF data. The self-identification parameters of the APS may include an APS ID. That is, in addition to the ALF data field, the APS may also include information indicating the APS ID. The patch group header or the slice header may reference the APS using the APS index information. In other words, the patch group header or the slice header may include the APS index information, and the ALF process for the target block may be performed based on the ALF data (ALF parameters) included in the APS having the APS ID indicated by the APS index information. Here, the APS index information may be referred to as the APS ID information.

[0153] In addition, the SPS may include a flag that allows the use of ALF. For example, when the CVS starts, the SPS may be checked, and the flag may be checked in the SPS. For example, the SPS may include the syntax of Table 1 below. The syntax of Table 1 may be part of the SPS.

[0154] [Table 1]

[0155]

[0156] For example, the semantics of the syntax elements included in the syntax of Table 1 may be specified as shown in the following table.

[0157] [Table 2]

[0158]

[0159] That is, the sps_alf_enabled_flag syntax element may indicate whether the ALF is available based on whether the value is 0 or 1. The sps_alf_enabled_flag syntax element may be referred to as an ALF availability flag (which may be referred to as a first ALF availability flag) and may be included in the SPS. That is, the ALF availability flag may be signaled at the SPS (or SPS level). When the value of the ALF availability flag signaled by the SPS is 1, it may be determined that the ALF is basically available for pictures in the CVS referring to the SPS. Meanwhile, as described above, the ALF may be turned on / off separately by signaling an additional availability flag at a level lower than the SPS.

[0160] For example, if the ALF tool is available for CVS, an additional availability flag (which may be referred to as a second ALF availability flag) may be signaled in the tiling header or slice header. For example, when ALF is available at the SPS level, the second ALF availability flag may be parsed / signaled. If the value of the second ALF availability flag is 1, the ALF data may be parsed via the tiling header or slice header. For example, the second ALF availability flag may specify the ALF availability conditions for luma and chroma components. The ALF data may be accessed via the APS ID information.

[0161] [Table 3]

[0162]

[0163] [Table 4]

[0164]

[0165] For example, the semantics of the syntax elements included in the syntax of Table 3 or Table 4 may be specified as shown in the following table.

[0166] [Table 5]

[0167]

[0168] [Table 6]

[0169]

[0170] The second ALF availability flag may include a tile_group_alf_enabled_flag syntax element or a slice_alf_enabled_flag syntax element.

[0171] Based on the APS ID information (eg, tile_group_aps_id syntax element or slice_aps_id syntax element), the APS referenced by the corresponding tile group or the corresponding slice may be identified. The APS may include ALF data.

[0172] Meanwhile, the structure of the APS including the ALF data may be described based on, for example, the following syntax and semantics. The syntax of Table 7 may be a part of the APS.

[0173] [Table 7]

[0174]

[0175] [Table 8]

[0176]

[0177] As described above, the adaptation_parameter_set_id syntax element may indicate an identifier of a corresponding APS. That is, the APS may be identified based on the adaptation_parameter_set_id syntax element. The adaptation_parameter_set_id syntax element may be referred to as APS ID information. Also, the APS may include an ALF data field. The ALF data field may be parsed / signaled after the adaptation_parameter_set_id syntax element.

[0178] In addition, for example, an APS extension flag (e.g., an aps_extension_flag syntax element) may be parsed / signaled in the APS. The APS extension flag may indicate whether an APS extension data flag (aps_extension_data_flag) syntax element is present. The APS extension flag may be used, for example, to provide an extension point for later versions of the VVC standard.

[0179] The core processing / handling of the ALF information may be performed in a slice header or a tile group header. The ALF data field may include information about the processing of the ALF filter. For example, the information that may be extracted from the ALF data field includes information about the number of filters used, information indicating whether the ALF is applied only to the luma component, information about the color components and exponential Golomb (EG) parameters and / or information about delta values ​​of filter coefficients, etc.

[0180] Meanwhile, the ALF data field may include, for example, the following ALF data syntax.

[0181] [Table 9]

[0182]

[0183] For example, the semantics of the syntax elements included in the syntax of Table 9 may be specified as shown in the following table.

[0184] [Table 10]

[0185]

[0186]

[0187]

[0188]

[0189] For example, the parsing of ALF data through a tile group header or a slice header may be started by first parsing / signaling an alf_chroma_idc syntax element. The alf_chroma_idc syntax element may have a value ranging from 0 to 3. The value may indicate whether the ALF-based filtering process is applied only to the luma component or to a combination of the luma component and the chroma component. Once the availability (available parameters) of each component is determined, information about the number of luma (component) filters may be parsed. As an example, the maximum number of available filters may be set to 25. If the number of luma filters signaled is at least one, for each filter ranging from 0 to the maximum number of filters (e.g., 25, which may alternatively be referred to as a category), the filter index information may be parsed / signaled. This may imply that each category (i.e., from 0 to the maximum number of filters) is associated with a filter index. When the filter to be used for each category is marked based on the filter index, a flag (e.g., alf_luma_coeff_delta_flag) may be parsed / signaled. The flag may be used to interpret whether flag information related to the prediction of the ALF luma filter coefficient delta value (eg, alf_luma_coeff_delta_prediction_flag) is present in a slice header or a tile group header.

[0190] If the number of luma filters signaled by the alf_luma_num_filters_signalled_minus1 syntax element is greater than 0, and the value of the alf_luma_coeff_delta_flag syntax element is 0, it may mean that the alf_luma_coeff_delta_prediction_flag syntax element is present in the slice header or the patch group header, and its state may be evaluated. If the state of the alf_luma_coeff_delta_prediction_flag syntax element indicates 1, it may mean that the luma filter coefficients are predicted from previous luma (filter) coefficients. If the state of the alf_luma_coeff_delta_prediction_flag syntax element indicates 0, it may mean that the luma filter coefficients are not predicted from deltas of previous luma (filter) coefficients.

[0191] When the delta filter coefficients (e.g., alf_luma_coeff_delta_abs) are coded based on exponential Golomb codes, it may be necessary to determine the order k of the exponential Golomb (EG) code in order to decode the delta luma filter coefficients (e.g., alf_luma_coeff_delta_abs). This information may be needed to decode the filter coefficients. The order of the exponential Golomb code may be denoted as EG(k). To determine EG(k), the alf_luma_min_eg_order_minus1 syntax element may be parsed / signaled. The alf_luma_min_eg_order_minus1 syntax element may be an entropy coded syntax element. The alf_luma_min_eg_order_minus1 syntax element may indicate the minimum order of EG for decoding delta luma filter coefficients. For example, the value of the alf_luma_min_eg_order_minus1 syntax element may be in the range of 0 to 6. After parsing / signaling the alf_luma_min_eg_order_minus1 syntax element, the alf_luma_eg_order_increase_flag syntax element may be parsed / signaled. If the value of the alf_luma_eg_order_increase_flag syntax element is 1, this indicates that the order of the EG indicated by the alf_luma_min_eg_order_minus1 syntax element is increased by 1. If the value of the alf_luma_eg_order_increase_flag syntax element is 0, this indicates that the order of the EG indicated by the alf_luma_min_eg_order_minus1 syntax element is not increased. The order of the EG may be represented by the index of the EG. For example, the EG order (or EG index) based on the alf_luma_min_eg_order_minus1 syntax element and the alf_luma_eg_order_increase_flag syntax element (related to the luma component) may be determined as follows.

[0192] [Table 11]

[0193]

[0194] Based on the above determination process, expGoderY can be derived as xpGoderY=KminTab. In this way, an array including EG orders can be derived, which can be used by a decoding device. expGoOrderY can indicate an EG order (or EG index).

[0195] There may be a predefined Golomb order index (ie, golombOrderIdxY). The predefined Golomb order may be used to determine the final Golomb order for coding the coefficients.

[0196] For example, the predefined Golomb order may be configured, for example, as in the following table.

[0197] golombOrderIdxY[]={0, 0, 1, 0, 1, 2, 1, 0, 0, 1, 2}

[0198] Here, the order k=expGoderY[GolombOrderIdxY[j]], and j can be indicated as the jth signaled filter coefficient. For example, if j=2, that is, the third filter coefficient golomborderIdxY[2]=1, then k=expGoderY[1].

[0199] In this case, for example, if the value of the alf_luma_coeff_delta_flag syntax element indicates true, ie 1, then for each filter that is signaled, an alf_luma_coeff_flag syntax element may be signaled. The alf_luma_coeff_flag syntax element indicates whether the luma filter coefficients are (explicitly) signaled.

[0200] When the EG order and the state of the above-mentioned related flags (e.g., alf_luma_coeff_delta_flag, alf_luma_coeff_flag, etc.) are determined, the difference information and sign information of the luma filter coefficient may be parsed / signaled (i.e., when alf_luma_coeff_flag indicates true). The delta absolute value information of each of the 12 filter coefficients may be parsed / signaled (alf_luma_coeff_delta_abs syntax element). In addition, if the alf_luma_coeff_delta_abs syntax element has a value, the sign information (alf_luma_coeff_delta_sign syntax element) may be parsed / signaled. Information including the difference information and sign information of the luma filter coefficient may be referred to as information about the luma filter coefficient.

[0201] The increment of the filter coefficient may be determined and stored with the sign. In this case, the increment of the filter coefficient with the sign may be stored in the form of an array, which may be denoted as filterCoefficient. The increment of the filter coefficient may be referred to as the incremental brightness coefficient, and the increment of the filter coefficient with the sign may be referred to as the incremental brightness coefficient with the sign.

[0202] To determine the final filter coefficients from the signed delta-luminance coefficients, the (luminance) filter coefficients may be updated as follows.

[0203] filterCoefficients[sigFiltIdx][j]+=filterCoefficients[sigFiltIdx][j]

[0204] Here, j may indicate a filter coefficient index, and sigFiltIdx may indicate a signaled filter index. j={0, ..., 11} and sigFiltIdx={0, ..., alf_luma_filters_signaled_minus1}.

[0205] The coefficients can be copied into the final AlfCoeffl[filtIdx][j]. Here, filtidx=0,...,24, and j=0,...,11.

[0206] The signed delta-luminance coefficient for a given filter index may be used to determine the first 12 filter coefficients. For example, the thirteenth filter coefficient for a 7×7 filter may be determined based on the following equation. The thirteenth filter coefficient may represent the center-tap filter coefficient described above.

[0207] [Equation 1]

[0208] Alf Coeff L [filtIdx]

[12] =128-∑ k Alf Coeff L [filtIdx][k]<<t

[0209] Here, the filter coefficient index 12 may indicate the thirteenth filter coefficient. For reference, since the filter coefficient index starts from 0, the value 12 may indicate the thirteenth filter coefficient.

[0210] For example, to ensure bitstream consistency, when k is 0, ..., 11, the value of the final filter coefficient AlfCoeffl[filtIdx][k] ranges from -2 7 To 2 7 -1, and when k is 12, it can range from 0 to 2 8 -1. Here, k can be replaced by j.

[0211] If processing for luma components is performed, processing for chroma components may be performed based on the alf_chroma_idc syntax element. If the value of the alf_chroma_idc syntax element is greater than 0, minimum EG order information for chroma components (e.g., alf_chroma_min_eg_order_minus1 syntax element) may be parsed / signaled. According to the above-described embodiment of the present document, a 5×5 diamond filter shape may be used for chroma components, and in this case, the maximum Golomb index may be 2. In this case, the EG order (or EG index) of the chroma component may be determined, for example, as follows.

[0212] [Table 12]

[0213]

[0214] Based on the determination process, expGoOrderC may be derived as expGoOrderC=KminTab. Thus, an array containing EG orders that can be used by the decoding apparatus can be derived. expGoOrderC may indicate the EG order (or EG index) of the chroma component.

[0215] There may be a predefined (predefined) Golomb order index (golombOrderIdxC). The predefined Golomb order may be used to determine the final Golomb order used to code the coefficients.

[0216] For example, the predefined Golomb order may be configured, for example, as in the following table.

[0217] golombOrderIdxC[]={0, 0, 1, 0, 0, 1}

[0218] Here, the order k=expGoOrderC[golombOrderIdxC[j]], and j may represent the jth signaled filter coefficient. For example, if j=2, ie, the third filter coefficient golomboorderIdxY[2]=1, then k=expGoOrderC[1].

[0219] Based on this, the absolute value information and sign information of the chroma filter coefficient can be parsed / signaled. Information including the absolute value information and sign information of the chroma filter coefficient can be referred to as information about the chroma filter coefficient. For example, a 5×5 diamond filter shape can be applied to the chroma component, and in this case, the absolute increment information (alf_chroma_coeff_abs syntax element) of each of the six (chroma component) filter coefficients can be parsed / signaled. In addition, if the value of the alf_chroma_coeff_abs syntax element is greater than 0, the sign information (alf_chroma_coeff_abs syntax element) can be parsed / signaled. For example, six chroma filter coefficients can be derived based on the information about the chroma filter coefficient. In this case, for example, the seventh chroma filter coefficient can be determined based on the following equation. The seventh filter coefficient can represent the above-mentioned center tap filter coefficient.

[0220] [Equation 2]

[0221] Alf Coeff C [6] = 128-∑ k Alf Coeff C [filtIdx][k]<<1

[0222] Here, the filter coefficient index 6 may indicate the seventh filter coefficient. For reference, since the filter coefficient index starts from 0, the value 6 may indicate the seventh filter coefficient.

[0223] For example, to ensure bitstream consistency), when k is 0, ..., 5, the value of the final filter coefficient AlfCoeffc[filtIdx][k] ranges from -2 7 To 2 7 -1, and when k is 6, it can range from 0 to 2 8 -1. Here, k can be replaced by j.

[0224] If (luminance / chrominance) filter coefficients are derived, ALF-based filtering can be performed based on the filter coefficients or filters including the filter coefficients. As described above, modified reconstructed samples can be derived in this way. In addition, multiple filters can be derived, and the filter coefficients of one of the multiple filters can be used for the ALF process. As an example, one of the multiple filters can be indicated based on filter selection information notified by signaling. Alternatively, for example, one of the multiple filters can be selected based on the activity and / or directionality of the current block or the ALF target block, and the filter coefficients of the selected filter can be used for the ALF process.

[0225] Meanwhile, as described above, Luma Mapping and Chroma Scaling (LMCS) may be applied to improve coding efficiency. LMCS may be referred to as a loop shaper (shaping). In order to increase coding efficiency, signaling of LMCS control and / or LMCS-related information may be performed hierarchically.

[0226] Fig. 9 The hierarchical structure of the CVS according to the embodiment of the present document is exemplarily shown. A coded video sequence (CVS) may include a sequence parameter set (SPS), a picture parameter set (PPS), a tiling group header, tiling data and / or a CTU. Here, the tiling group header and the tiling data may be referred to as a slice header and slice data, respectively.

[0227] The SPS may originally include a flag for enabling tools to enable them to be used in the CVS. In addition, the SPS may be referenced by a PPS including information about parameters that change for each picture. Each of the coded pictures may include one or more tiles of a coded rectangular domain. The tiles may be grouped to form a tile group raster scan. Each tile group is encapsulated with header information called a tile group header. Each tile consists of a CTU containing coded data. Here, the data may include original sample values, predicted sample values, and their brightness and chrominance components (brightness predicted sample values ​​and chrominance predicted sample values).

[0228] Fig.10 An exemplary LMCS structure according to an embodiment of this document is shown. Fig.10 The LMCS structure 1000 may include an in-loop mapping portion 1010 for luma components based on an adaptive piecewise linear (adaptive PWL) model and a luma-dependent chroma residual scaling portion 1020 for chroma components. The blocks of dequantization and inverse transformation 1011, reconstruction 1012, and intra prediction 1013 of the in-loop mapping portion 1010 represent the processing applied in the mapped (shaped) domain. The blocks of loop filter 1015, motion compensation or inter prediction 1017 of the in-loop mapping portion 1010, and the blocks of reconstruction 1022, intra prediction 1023, motion compensation or inter prediction 1024, and loop filter 1025 of the chroma residual scaling portion 1020 represent the processing applied in the original (non-mapped, non-shaped) domain.

[0229] like Fig.10As shown, when LMCS is enabled, at least one of an inverse shaping (mapping) process 1014, a forward shaping (mapping) process 1018, and a chroma scaling process 1021 may be applied. For example, the inverse shaping process may be applied to a (reconstructed) luma sample (or multiple luma samples or a luma sample array) of a reconstructed picture. The inverse shaping process may be performed based on a piecewise function (inverse) index of the luma sample. The piecewise function (inverse) index may identify the segment (or portion) to which the luma sample belongs. The output of the inverse shaping process is a modified (reconstructed) luma sample (or multiple modified luma samples or a modified luma sample array). LMCS may be enabled or disabled at a tile group (or slice), picture, or higher level.

[0230] A forward shaping process and / or a chroma scaling process may be applied to generate a reconstructed picture. A picture may include luma samples and chroma samples. A reconstructed picture with luma samples may be referred to as a reconstructed luma picture, and a reconstructed picture with chroma samples may be referred to as a reconstructed chroma picture. A combination of a reconstructed luma picture and a reconstructed chroma picture may be referred to as a reconstructed picture. A reconstructed luma picture may be generated based on a forward shaping process. For example, if inter prediction is applied to a current block, forward shaping is applied to luma prediction samples derived based on (reconstructed) luma samples of a reference picture. Since the (reconstructed) luma samples of the reference picture are generated based on an inverse shaping process, forward shaping may be applied to the luma prediction samples to derive shaped (mapped) luma prediction samples. A forward shaping process may be performed based on a piecewise function index of the luma prediction samples. The piecewise function index may be derived based on a value of a luma prediction sample of a reference picture used for inter prediction or a value of a luma sample. A reconstructed sample may be generated based on the (shaped / mapped) luma prediction sample. An inverse shaping (mapping) process may be applied to the reconstructed sample. Reconstructed samples to which the inverse shaping (mapping) process has been applied may be referred to as inverse shaped (mapped) reconstructed samples. In addition, inverse shaped (mapped) reconstructed samples may be simply referred to as shaped (mapped) reconstructed samples. When intra prediction (or intra block copy (IBC)) is applied to the current block, forward mapping of the prediction samples of the current block may not be required because the inverse shaping process has not yet been applied to the reconstructed samples with reference to the current picture. In a reconstructed luma picture, luma samples may be generated (reconstructed) based on the (shaped) luma prediction samples and the corresponding luma residual samples.

[0231] A reconstructed chroma picture may be generated based on a chroma scaling process. For example, a reconstructed chroma picture may be generated based on chroma prediction samples and chroma residual samples (c res ) to derive the (reconstructed) chroma samples in the reconstructed chroma picture. Based on the (scaled) chroma residual samples (c resScale ) and the chroma residual scaling factor (cScaleInv can be called varScale) to derive the chroma residual samples (c resThe chroma residual scaling factor may be calculated based on the shaped luma prediction sample value in the current block. For example, the chroma residual scaling factor may be calculated based on the shaped luma prediction sample value (Y' pred )'s average brightness value (ave(Y' pred )) to calculate the scaling factor. For reference, based on Fig.16 The (scaled) chroma residual samples derived by the inverse transform / dequantization in may be referred to as c rescale , and the chroma residual samples derived by performing the (inverse) scaling process on the (scaled) chroma residual samples may be referred to as c res .

[0232] Fig.11 The LMCS structure according to another embodiment of this document is shown. Fig.10 describe Fig.11 Here, we will mainly describe Fig.11 The LMCS structure and Fig.10 The difference between the LMCS structures. Fig.11 The in-loop mapping part and the luminance-dependent chrominance residual scaling part can be compared with Fig.10 The in-loop mapping part and the luma-dependent chroma residual scaling part operate in the same way.

[0233] refer to Fig.11 , the chroma residual scaling factor can be derived based on the luma reconstruction samples. In this case, the average luma value (avgY) can be obtained based on the adjacent luma reconstruction samples outside the reconstruction block instead of the luma reconstruction samples inside the reconstruction block. r ), and can be based on the average brightness value (avgY r ) to derive the chroma residual scaling factor. Here, the adjacent luma reconstruction samples may be adjacent luma reconstruction samples of the current block, or may be adjacent luma reconstruction samples of a virtual pipeline data unit (VPDU) including the current block. For example, when intra prediction is applied to the target block, the reconstructed samples may be derived based on the prediction samples, wherein the prediction samples are derived based on the intra prediction. In addition, for example, when inter prediction is applied to the target block, forward mapping is applied to the prediction samples derived based on the inter prediction, and the reconstructed samples may be generated based on the shaped (or forward mapped) luma prediction samples.

[0234] The motion picture / image information signaled by the bitstream may include LMCS parameters (information about LMCS). The LMCS parameters may be configured as high-level syntax (HLS, including slice header syntax), etc. A detailed description of the LMCS parameters and configuration will be described later. As described above, the syntax table described in this document (and the following embodiments) may be constructed / encoded on the encoder (encoding device) side and signaled to the decoder (decoding device) via the bitstream. The decoder may parse / decode the information about LMCS in the syntax table (in the form of syntax components). One or more embodiments described below may be combined. The encoder may encode the current image based on the information about LMCS, and the decoder may decode the current image based on the information about LMCS.

[0235] The in-loop mapping of the luminance component can adjust the dynamic range of the input signal to improve compression efficiency by redistributing codewords across the dynamic range. For luminance mapping, a forward mapping (shaping) function (FwdMap) and an inverse mapping (shaping) function (InvMap) corresponding to the forward mapping function (FwdMap) can be used. The forward mapping function (FwdMap) can be signaled using a partial linear model, for example, the partial linear model can have 16 segments or bins. These segments can have the same length. In one example, the inverse mapping function (InvMap) may not be signaled separately, but can be derived from the forward mapping function (FwdMap). That is, the inverse mapping can be a function of the forward mapping. For example, the inverse mapping function can be a function in which the forward mapping function is symmetric based on y=x.

[0236] In-loop (luminance) shaping can be used to map input luminance values ​​(samples) to changed values ​​in the shaping domain. After reconstruction, the shaped values ​​can be encoded and mapped back to the original (unmapped, unshaped) domain. Chroma residual scaling can be applied to compensate for the difference between the luminance signal and the chrominance signal. In-loop shaping can be performed by specifying a high-level syntax for the shaper model. The shaper model syntax can signal a partial linear model (PWL model). A forward lookup table (FwLUT) and / or an inverse lookup table (InvLUT) can be derived based on the partial linear model. As an example, when the forward lookup table FwLUT is derived, the inverse lookup table InvLUT can be derived based on the forward lookup table FwLUT. The forward lookup table FwLUT can map the input luminance value Yi to the changed value Yr, and the inverse lookup table InvLUT can map the reconstructed value Yr to the reconstructed value Y'i based on the changed value.

[0237] In one example, the SPS may include the syntax of Table 13 below. The syntax of Table 13 may include sps_reshaper_enabled_flag as a tool enable flag. Here, sps_reshaper_enabled_flag may be used to specify whether a reshaper is used in a coded video sequence (CVS). That is, sps_reshaper_enabled_flag may be a flag for enabling reshaping in the SPS. In one example, the syntax of Table 13 may be part of the SPS.

[0238] [Table 13]

[0239]

[0240] In one example, the semantics that may be indicated by sps_seq_parameter_set_id and sps_reshaper_enabled_flag may be as shown in Table 14 below.

[0241] [Table 14]

[0242]

[0243] In one example, the tile group header or the slice header may include the syntax of Table 15 or Table 16 below.

[0244] [Table 15]

[0245]

[0246] [Table 16]

[0247]

[0248] The semantics of the syntax elements included in the syntax of Table 15 or Table 16 may include, for example, the contents disclosed in the following table.

[0249] [Table 17]

[0250]

[0251] [Table 18]

[0252]

[0253] As an example, if sps_reshaper_enabled_flag is parsed, additional data (e.g., information included in Table 15 or 16 above) for configuring the lookup tables (FwdLUT and / or InvLUT) may be parsed in the tile group header. To this end, the state of the SPS reshaper flag may be checked in the slice header or the tile group header. When sps_reshaper_enabled_flag is true (or 1), the additional flag tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) may be parsed. The purpose of tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) may be to indicate the presence of a reshaper model. For example, when tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) is true (or 1), it may indicate that a reshaper is present for the current tile group (or current slice). When tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) is false (or 0), it may indicate that no reshaper exists for the current tile group (or current slice).

[0254] If a reshaper exists in the current tile group (or current slice) and is enabled, a reshaper model may be processed (e.g., tile_group_reshaper_model() or slice_reshaper_model()), and tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) may be parsed in addition to additional flags. The tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) may indicate whether the reshaper model is used for the current tile group (or current slice). For example, if tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is 0 (or false), it may indicate that the reshaper model is not used for the current tile group (or current slice). If tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is 1 (or true), it may indicate that the reshaper model is used for the current tile group (or current slice).

[0255] As an example, for example, tile_group_reshaper_model_present_flag (or slice_reshaper_model_present_flag) may be true (or 1), and tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) may be false (or 0). This means that the reshaper model is present but not used in the current tile group (or current slice). In this case, the reshaper model may be used in the following tile groups (or slices). As another example, tile_group_reshaper_enable_flag may be true (or 1) and tile_group_reshaper_model_present_flag may be false (or 0).

[0256] When parsing a reshaper model (e.g., tile_group_reshaper_model() or slice_reshaper_model()) and tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag), it may be determined (evaluated) whether conditions required for chroma scaling exist. The above conditions may include condition 1 (the current tile group / slice is not intra-coded) and / or condition 2 (the current tile group / slice is not partitioned into two separate coded quadtree structures for luma and chroma, i.e., the current tile group / slice is not a binary tree structure). If condition 1 and / or condition 2 are true and / or tile_group_reshaper_enable_flag (or slice_reshaper_enable_flag) is true (or 1), tile_group_reshaper_chroma_residual_scale_flag (or slice_reshaper_chroma_residual_scale_flag) may be parsed. When tile_group_reshaper_chroma_residual_scale_flag (or slice_reshaper_chroma_residual_scale_flag) is enabled (if 1 or true), it may indicate that chroma residual scaling is enabled for the current tile group (or current slice). When tile_group_reshaper_chroma_residual_scale_flag (or slice_reshaper_chroma_residual_scale_flag) is disabled (if 0 or false), it may indicate that chroma residual scaling is disabled for the current tile group (or current slice).

[0257] The purpose of the above shaping is to parse the data required to construct the lookup table (FwdLUT and / or InvLUT). In an example, the lookup table constructed based on the parsed data can divide the distribution of the allowable brightness value range into multiple bins (e.g., 16). Therefore, the brightness value in a given bin can be mapped to a changed brightness value.

[0258] Fig.12 A diagram showing an exemplary forward mapping is shown. Fig.12 In FIG. 5 , five bins are shown by way of example only.

[0259] refer to Fig.12, the x-axis represents the input luma value, and the y-axis represents the changed output luma value. The x-axis is divided into 5 bins or segments, each bin having a length of L. That is, the five bins mapped to the changed luma value have the same length. The forward lookup table (FwdLUT) may be constructed using data available in the tile group header (e.g., shaper data) from which the mapping is facilitated.

[0260] In an embodiment, an output pivot point associated with a bin index may be calculated. The output pivot point may set (mark) the minimum and maximum boundaries of the output range of the brightness codeword shaping. The process of calculating the output pivot point may be performed based on a piecewise cumulative distribution function of the number of codewords. The output pivot range may be divided based on the maximum number of bins to be used and the size of the lookup table (FwdLUT or InvLUT). As an example, the output pivot range may be divided based on the product between the maximum number of bins and the size of the lookup table. For example, if the product between the maximum number of bins and the size of the lookup table is 1024, the output pivot range may be divided into 1024 entries. The division of the output pivot range may be performed (applied or implemented) based on (using) a scaling factor. In one example, the scaling factor may be derived based on equation 3 below.

[0261] [Equation 3]

[0262] SF=(y2-y1)*(1<<FP_PREC)+c

[0263] In Equation 3, SF represents a scaling factor, and y1 and y2 represent output pivot points corresponding to respective bins. In addition, FP_PREC and c may be predetermined constants. The scaling factor determined based on Equation 3 may be referred to as a scaling factor for forward shaping.

[0264] In another embodiment, for inverse shaping (inverse mapping), for a defined range of bins (e.g., reshaper_model_min_bin_idx to reshaper_model_max_bin_idx), the input shaping pivot point corresponding to the mapped pivot point of the forward lookup table (FwdLUT) and the mapped inverse output pivot point (given as the initial codeword number * bin index) are obtained. In another example, the scaling factor SF may be derived based on the following equation 4.

[0265] [Equation 4]

[0266] SF=(y2-y1)*(1<<FP_PREC) / (x2-x1)

[0267] In Equation 4, SF represents a scaling factor, x1 and x2 represent input pivot points, and y1 and y2 represent output pivot points corresponding to each segment (bin). Here, the input pivot point may be a pivot point mapped based on a forward lookup table (FwdLUT), and the output pivot point may be a pivot point reversely mapped based on an inverse lookup table (InvLUT). In addition, FP_PREC may be a predetermined constant. The FP_PREC of Equation 4 may be the same as or different from the FP_PREC of Equation 3, and the scaling factor determined based on Equation 4 may be referred to as a scaling factor for inverse shaping. During inverse shaping, a partition of the input pivot point may be performed based on the scaling factor of Equation 4, and based on the partitioned input pivot point, a pivot value corresponding to the minimum and maximum bin values ​​of a bin index from 0 to a minimum bin index (reshaper_model_min_bin_idx) and / or from a minimum bin index (reshaper_model_min_bin_idx) to a maximum bin index (reshaper_model_max_bin_idx) is specified.

[0268] Table 19 below shows the syntax of the shaper model according to an embodiment. The shaper model may be referred to as a LMCS model. Here, the shaper model is exemplarily described as a tile group shaper, but the present specification is not necessarily limited to this embodiment. For example, the shaper model may be included in the APS, or the tile group shaper model may be referred to as a slice shaper model.

[0269] [Table 19]

[0270]

[0271] The semantics of the syntax elements included in the syntax of Table 19 may include, for example, the contents disclosed in the following table.

[0272] [Table 20]

[0273]

[0274]

[0275] The reshape model includes components such as reshape_model_min_bin_idx, reshape_model_delta_max_bin_idx, reshaper_model_bin_delta_abs_cw_prec_minus1, reshape_model_bin_delta_abs_CW[i], and reshaper_model_bin_delta_sign_CW as elements. Each component will be described in detail below.

[0276] reshape_model_min_bin_idx indicates the minimum bin (or segment) index used during the reshaper configuration process. The value of reshape_model_min_bin_idx can be in the range of 0 to MaxBinIdx. For example, MaxBinIdx can be 15.

[0277] In an embodiment, the patch group reshaper model may preferably parse two indices (or parameters), namely, reshaper_model_min_bin_idx and reshaper_model_delta_max_bin_idx. The maximum bin index (reshaper_model_max_bin_idx) may be derived (determined) based on these two indices. reshaper_model_delta_max_bin_idx may represent the maximum bin index MaxBinIdx allowed minus the actual maximum bin index (reshaper_model_max_bin_idx) used in the reshaper configuration process. The value of the maximum bin index (reshaper_model_max_bin_idx) may be in the range from 0 to MaxBinIdx. For example, MaxBinIdx may be 15. As an example, the value of reshaper_model_max_bin_idx may be derived based on Equation 5 below.

[0278] [Equation 5]

[0279] reshape_model_max_bin_idx=MaxBinldx-reshape_model_delta_max_bin_idx.

[0280] The maximum bin index (reshaper_model_max_bin_idx) may be greater than or equal to the minimum bin index (reshaper_model_min_bin_idx). The minimum bin index may be referred to as the minimum allowed bin index or the minimum allowed bin index, and the maximum bin index may also be referred to as the maximum allowed bin index or the maximum allowed bin index.

[0281] If the maximum bin index (reshaper_model_max_bin_idx) is derived, the syntax element reshaper_model_bin_delta_abs_cw_prec_minus1 may be parsed. The number of bits used to represent the syntax reshape_model_bin_delta_abs_CW[i] may be determined based on reshaper_model_bin_delta_abs_cw_prec_minus1. For example, the number of bits used to represent reshape_model_bin_delta_abs_CW[i] may be the same as reshaper_model_bin_delta_abs_cw_prec_minus1 plus 1.

[0282] reshape_model_bin_delta_abs_CW[i] may indicate information related to the absolute delta codeword value (absolute value of the delta codeword) of the i-th bin. In one example, if the absolute delta codeword value of the i-th bin is greater than 0, reshaper_model_bin_delta_sign_CW_flag[i] may be parsed. The sign of reshape_model_bin_delta_abs_CW[i] may be determined based on reshaper_model_bin_delta_sign_CW_flag[i]. In one example, if reshaper_model_bin_delta_sign_CW_flag[i] is 0 (or false), the corresponding variable RspDeltaCW[i] may be positive. In other cases (if reshaper_model_bin_delta_sign_CW_flag[i] is not 0, if reshaper_model_bin_delta_sign_CW_flag[i] is 1 (or true)), the corresponding variable RspDeltaCW[i] may be negative. If reshaper_model_bin_delta_sign_CW_flag[i] is not present, it may be inferred to be 0 (or false).

[0283] In an embodiment, the variable RspDeltaCW[i] may be derived based on the above reshape_model_bin_delta_abs_CW[i] and reshape_model_bin_delta_sign_CW_flag[i]. RspDeltaCW[i] may be referred to as the value of the delta codeword. For example, RspDeltaCW[i] may be derived based on the following equation 6.

[0284] [Equation 6]

[0285] RspDeltaCW[i] = (1 - 2 * reshape_model_bin_delta_sign_CW[i]) * reshape_model_bin_delta_abs_CW[i]

[0286] In Equation 6, reshape_model_bin_delta_sign_CW[i] can be information related to the sign of RspDeltaCW[i]. For example, reshape_model_bin_delta_sign_CW[i] can be the same as the above-mentioned reshaper_model_bin_delta_sign_CW_flag[i]. Here, i can range from the minimum bin index (reshaper_model_min_bin_idx) to the maximum bin index (reshape_model_max_bin_idx).

[0287] The variable (or array) RspCW[i] can be derived based on RspDeltaCW[i]. Here, RspCW[i] can represent the number of codewords assigned (distributed) to the i-th bin. That is, the number of codewords assigned (distributed) to each bin can be stored in array form. In one example, if i is less than the above-mentioned reshaper_model_min_bin_idx or greater than reshaper_model_max_bin_idx, (i < reshaper_model_min_bin_idx or reshaper_model_max_bin_idx < i), then RspCW[i] can be 0. Otherwise (if i is greater than or equal to the above-mentioned reshaper_model_min_bin_idx and less than or equal to reshaper_model_max_bin_idx, (reshaper_model_min_bin_idx <= i <= reshaper_model_max_bin_idx), then RspCW[i] can be derived based on the above-mentioned RspDeltaCW[i], the luminance bit depth (BitDepthsy), and / or MaxBinIdx. In this case, for example, RspCW[i] can be derived based on the following Equation 7.

[0288] [Equation 7]

[0289] RspCW[i] = OrgCW + RspDeltaCW[i]

[0290] In Equation 7, OrgCW may be a predetermined value, for example, may be determined based on Equation 8 below.

[0291] [Equation 8]

[0292] OrgCW=(1<<BitDepth γ ) / (MaxBinldx+1)

[0293] In Equation 8, BitDepthY is the luma bit depth, and MaxBinIdx represents the maximum allowable bin index. In one example, if BitDepthY is 10, RspCW[i] can have values ​​from 32 to 2*OrgCW-1.

[0294] The variable InputPivot[i] may be derived based on the above-mentioned OrgCW. For example, InputPivot[i] may be derived based on the following equation 9.

[0295] [Equation 9]

[0296] InputPivot[i]=i*OrgCW

[0297] The variables ReshapePivot[i], ScaleCoef[i] and / or InvScaleCoeff[i] can be derived based on the above-mentioned RspCW[i], InputPivot[i] and / or OrgCW, for example, ReshapePivot[i], ScaleCoef[i] and / or InvScaleCoeff[i] can be derived based on the following Table 21.

[0298] [Table 21]

[0299]

[0300] In Table 21, a for loop syntax may be used, where i increases from 0 to MaxBinIdx, and shiftY may be a predetermined constant for bit shifting. Whether InvScaleCoeff[i] is derived based on RspCW[i] may be determined based on a conditional clause depending on whether RspCW[i] is 0.

[0301] ChromaScaleCoef[i] used to derive the chroma residual scaling factor can be derived based on the following Table 22.

[0302] [Table 22]

[0303]

[0304] In Table 22, ShiftC may be a predetermined constant for bit shifting. Referring to Table 22, whether ChromaScaleCoef[i] is derived based on the array ChromaResidualScalLut may be determined based on a conditional clause according to whether RspCW[i] is 0. Here, ChromaResidualScalLut may be a predetermined array. However, the ChromaResidualScalLeLut array is merely exemplary, and the present embodiment is not necessarily limited to Table 22.

[0305] The method for deriving the i-th variable has been described above. The i+1-th variable may be based on ReshapePivot[i+1], and for example, ReshapePivot[i+1] may be derived based on Equation 10.

[0306] [Equation 10]

[0307] ReshapePivot[i+1]=ReshapePivot[i]+RspCW[i]

[0308] In equation 10, RspCW[i] may be derived based on equations 7 and / or 8 above. Luma mapping may be performed based on the above embodiments and examples, and the above syntax and components included therein may be merely exemplary representations, and the embodiments are not limited to the above tables or equations. Hereinafter, a method for performing chroma residual scaling (scaling of chroma components of residual samples) based on luma mapping will be described.

[0309] (Lumence-dependent) chroma residual scaling is to compensate for the difference between luma samples and corresponding chroma samples. For example, whether chroma residual scaling is enabled may be signaled at the tile group level or the slice level. In one example, if luma mapping is enabled and dual-tree partitioning is not applied to the current tile group, an additional flag may be signaled to indicate whether luma-dependent chroma residual scaling is enabled. In another example, if luma mapping is not used, or if dual-tree partitioning is not used for the current tile group, luma-dependent chroma residual scaling may be disabled. In another example, chroma residual scaling may be always disabled for chroma blocks having a size less than or equal to 4.

[0310] Chroma residual scaling may be performed based on an average luminance value of a reference sample. For example, the reference sample may include samples of a corresponding luminance prediction block (luminance component of a prediction block to which intra prediction and / or inter prediction is applied). When inter prediction is applied, the reference sample may include samples after forward mapping is applied to the luminance component prediction sample. Alternatively, the reference sample may include neighboring samples of the current block or neighboring samples of a VPDU including the current block. In this case, when inter prediction is applied to a neighboring block including neighboring samples, the neighboring samples may include luminance component reconstructed samples derived based on luminance component prediction samples to which forward mapping of the neighboring block is applied. The scaling operation on the encoder side and / or the decoder side may be implemented as a fixed-point integer calculation, for example, based on the following equation 11.

[0311] [Equation 11]

[0312] c'=sign(c)*((abs(c)*s+2CSCALE_FP_PREC-1)>>CSCALE_FP_PREC)

[0313] In the above Equation 11, c' represents a scaled chroma residual sample (a scaled chroma component of the residual sample), c represents a chroma residual sample (a chroma component of the residual sample), s may represent a chroma residual scaling factor, and CSCALE_FP_PREC may represent a predetermined constant.

[0314] As described above, the average luma value of the reference sample may be obtained, and the chroma residual scaling factor may be derived based on the average luma value. As described above, the chroma component residual samples may be scaled based on the chroma residual scaling factor, and the chroma component reconstructed samples may be generated based on the scaled chroma component residual samples.

[0315] An embodiment of the present document proposes a signaling structure for effectively applying the above-mentioned ALF and / or LMCS. According to an embodiment of the present document, for example, ALF data and / or LMCS data may be included in an HLS (e.g., APS), and by signaling a reference APS ID via lower-level header information (e.g., picture header, slice header) as an APS, a filter for an ALF and / or LMCS model (shaper model) may be adaptively derived. The LMCS model may be derived based on LMCS parameters. In addition, for example, multiple APS IDs may be signaled via header information, and in this way, different ALF and / or LMCS models may be applied in units of blocks within the same picture / slice.

[0316] For example, the following table shows an example of header information according to the embodiment.

[0317] [Table 23]

[0318]

[0319] The semantics of the syntax elements included in the syntax of Table 23 may include, for example, the contents disclosed in the following table.

[0320] [Table 24]

[0321]

[0322]

[0323]

[0324] For example, referring to Table 23, the header information may include ALF related information. The ALF related information includes an ALF availability flag syntax element (e.g., slice_ALF_enabled_flag), an ALF related APS ID number syntax element (e.g., num_alf_aps_ids_minus1), and an ALF related APS ID syntax element (e.g., slice_alf_aps_id[i], the number of which is the same as the number of ALF related APS IDs derived based on the ALF related APS ID number syntax element).

[0325] In addition, the header information may include LMCS related information. The LMCS related information may include, for example, at least one of the following: an LMCS availability flag syntax element (eg, slice_lmcs_enabled_flag), an LMCS related APS ID syntax element (slice_lmcs_aps_id), and a chroma residual scaling flag syntax element (slice_chroma_residual_scale_flag).

[0326] According to this embodiment, hierarchical control of ALF and / or LMCS is possible.

[0327] For example, as described above, the availability of the ALF tool may be determined by an ALF availability flag (e.g., sps_alf_enabled_flag) in an SPS, and then whether the ALF is available in the current picture or slice may be indicated by an ALF availability flag (e.g., slice_alf_enabeld_flag) in header information. When the value of the ALF availability flag in the header information is 1, an ALF-related APS ID number syntax element may be parsed / signaled. In addition, as many ALF-related APS ID syntax elements as the number of ALF-related APS IDs derived based on the ALF-related APS ID number syntax element may be parsed / signaled. That is, this may indicate that multiple APSs may be parsed or referenced by one header information.

[0328] Similarly, for example, whether the LMCS (or reshaping) tool is available can be determined by the LMCS availability flag (e.g., sps_reshaper_enabled_flag) in the SPS. sps_reshaper_enabled_flag can be referred to as sps_lmcs_enabled_flag. Whether the LMCS is available in the current picture or slice can be indicated by the LMCS availability flag (e.g., slice_lmcs_enabled_flag) in the header information. When the value of the LMCS availability flag in the header information is 1, the LMCS-related APS ID syntax element can be parsed / signaled. The LMCS model (shaper model) can be derived from the APS indicated by the LMCS-related APS ID syntax element. For example, the APS may also include an LMCS data field, and the LMCS data field may include the above-mentioned LMCS model (shaper model) information.

[0329] The following figures are created to explain specific examples of this specification. Since the names of specific devices or the names of specific signals / messages / fields described in the figures are presented by way of example, the technical features of this specification are not limited to the specific names used in the following figures.

[0330] Fig.13 and Fig.14 An example of a video / image encoding method and related components according to an embodiment of this document is schematically shown. Fig.13 The method disclosed in can be Figure 2 Specifically, for example, Fig.13 S1300 may be executed by the adder 250 of the encoding device, Fig.13 S1310 may be performed by the filter 260 of the encoding device, and Fig.13 S1320 may be performed by the entropy encoder 240 of the encoding device. Fig.13 The methods disclosed in may include the above-mentioned embodiments in this document.

[0331] Reference Fig.13, the encoding device generates a reconstructed sample of the current block in the current picture (S1300). The encoding device may derive the predicted sample of the current block based on the prediction mode. In this case, various prediction methods disclosed in this document (such as inter-frame prediction or intra-frame prediction) may be applied. The residual sample may be derived based on the predicted sample and the original sample. In this case, the residual information may be derived based on the residual sample. Based on the residual information, a (modified) residual sample may be derived. A reconstructed sample may be generated based on the (modified) residual sample and the predicted sample. A reconstructed block and a reconstructed picture may be derived based on the reconstructed sample.

[0332] The encoding device generates ALF related information (S1310). The encoding device derives ALF related ALF parameters, which can be applied to filter the reconstructed samples, and generates ALF related information. For example, the ALF related information may include the ALF related information described above in this document.

[0333] The encoding device encodes the image / video information (S1320). The image / video information may include information for generating reconstructed samples and / or ALF related information. The information for generating reconstructed samples may include, for example, prediction related information, residual information, and / or LMCS related information. The prediction related information may include information about various prediction modes (e.g., merge mode, MVP mode, etc.), MVD information, etc.

[0334] The encoded image / video information can be output in the form of a bit stream. The bit stream can be sent to a decoding device via a network or storage medium.

[0335] According to an embodiment of this document, the image / video information may include various information. For example, the image / video information may include information disclosed in at least one of Tables 1, 3, 4, 7, 9, 13, 15, 16, 19, and 23 above.

[0336] For example, the image information may include the above-mentioned Adaptation Parameter Set (APS). The APS may include ALF data, and the ALF data may include information for deriving filter coefficients for the ALF.

[0337] The image information may include an SPS, and the SPS may include a first ALF availability flag indicating whether the ALF is available. The image information may include header information. When the value of the first ALF availability flag is 1, the header information may include a second ALF availability flag indicating whether the ALF is available in a picture or a slice.

[0338] The header information may include an ALF-related APS ID syntax element, and the APS ID indicated by the ALF-related APS ID syntax element may be the same as an ID of an APS including the ALF data.

[0339] The header information may include a picture header or a slice header.

[0340] The header information includes a second ALF availability flag indicating whether ALF is available in a picture or a slice, and when the value of the second ALF availability flag is 1, the header information includes an ALF-related APS ID number syntax element, and the value of the ALF-related APS ID number syntax element plus 1 can be equal to the number of ALF-related APS IDs.

[0341] For example, the APS may include LMCS data. The header information may include LMCS related information. The LMCS related information may include, for example, at least one of the following: an LMCS availability flag syntax element (e.g., slice_lmcs_enabled_flag), an LMCS related APS ID syntax element (slice_lmcs_aps_id), and a chroma residual scaling flag syntax element (slice_chroma_residual_scale_flag).

[0342] Fig.15 and Fig.16 An example of an image / video decoding method and related components according to an embodiment of this document is schematically shown. Fig.15 The method disclosed in can be Figure 3 Specifically, for example, Fig.15 S1500 may be performed by the entropy decoder 310 of the decoding device, S1510 may be performed by the adder 340 of the decoding device, and S1520 may be performed by the filter 350 of the decoding device. Fig.15 The methods disclosed in may include the above-mentioned embodiments in this document.

[0343] Reference Fig.15 , the decoding device receives / obtains image / video information (S1500). The decoding device may receive / obtain image / video information through a bit stream.

[0344] According to an embodiment of this document, the image / video information may include various information. For example, the image / video information may include information disclosed in at least one of Tables 1, 3, 4, 7, 9, 13, 15, 16, 19, and / or 23 above.

[0345] The decoding device generates a reconstructed sample based on the image / video information (S1510). The decoding device may derive a prediction sample of the current block based on the prediction related information included in the image / video information. The decoding device may derive a residual sample based on the residual information included in the image / video information. The decoding device may generate a reconstructed sample based on the prediction sample and the residual sample. A reconstructed block and a reconstructed picture may be derived based on the reconstructed sample.

[0346] The decoding device performs an ALF process on the reconstructed sample (S1520). The decoding device may derive filter coefficients for the ALF and generate modified reconstructed samples based on the reconstructed sample and the filter coefficients. A filter may include a set of filter coefficients. The filter or filter coefficients may be derived based on the ALF data.

[0347] For example, the image information may include the above-mentioned APS (Adaptation Parameter Set). The APS may include ALF data, and the ALF data may include information for deriving filter coefficients for ALF.

[0348] The image information may include an SPS, and the SPS may include a first ALF availability flag indicating whether the ALF is available. The image information may include header information. When the value of the first ALF availability flag is 1, the header information may include a second ALF availability flag indicating whether the ALF is available in a picture or a slice.

[0349] The header information may include an ALF-related APS ID syntax element, and the APS ID indicated by the ALF-related APS ID syntax element may be the same as an ID of an APS including the ALF data.

[0350] The header information may include a picture header or a slice header.

[0351] The header information includes a second ALF availability flag indicating whether the ALF is available in the picture or slice, and when the value of the second ALF availability flag is 1, the header information may include an ALF-related APS ID number syntax element. The number of ALF-related APS IDs may be determined based on the value of the ALF-related APS ID number syntax element. This may indicate the number of ALF-related APS ID syntax elements included in the header information. For example, the value of the ALF-related APS ID number syntax element plus 1 may be equal to the number of ALF-related APS ID syntax elements included in the header information. For example, the ALF data may be included in at least one APS among the APS IDs indicated by the ALF-related APS ID syntax element.

[0352] For example, the APS may include LMCS data. The header information may include LMCS related information. The LMCS related information may include, for example, at least one of the following: an LMCS availability flag syntax element (e.g., slice_lmcs_enabled_flag), an LMCS related APS ID syntax element (slice_lmcs_aps_id), and a chroma residual scaling flag syntax element (slice_chroma_residual_scale_flag).

[0353] Although the method has been described based on a flowchart that lists steps or blocks in sequence in the above-mentioned embodiments, the steps of the embodiments are not limited to a specific order, and specific steps may be performed in a different order or at the same time than the above-mentioned steps. In addition, it will be understood by those of ordinary skill in the art that the steps of the flowchart are not exclusive, and another step may be included therein or one or more steps in the flowchart may be deleted without affecting the scope of the embodiments of this document.

[0354] The above-mentioned method according to the embodiment of the present document may be implemented in the form of software, and the encoding device and / or decoding device according to the present document may be included in a device that performs image processing such as a TV, a computer, a smart phone, a set-top box, and a display device.

[0355] When the embodiment is implemented with software in this document, the aforementioned method can be implemented by a module (process or function) that performs the aforementioned functions. The module can be stored in a memory and executed by a processor. The memory can be installed inside or outside the processor and can be connected to the processor via various known means. The processor may include an application specific integrated circuit (ASIC), other chipsets, logic circuits and / or data processing devices. The memory may include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium and / or other storage devices. In other words, the embodiments disclosed herein can be implemented and executed on a processor, a microprocessor, a controller or a chip. For example, the functional units illustrated in each figure can be implemented and executed on a computer, a processor, a microprocessor, a controller or a chip. In this case, information about the implementation (e.g., information about instructions) or an algorithm can be stored in a digital storage medium.

[0356] In addition, the decoding device and encoding device to which the embodiments of this document are applied may be included in the following: a multimedia broadcast transceiver, a mobile communication terminal, a home theater video device, a digital theater video device, a surveillance camera, a video chat device, and a real-time communication device such as video communication, a mobile streaming device, a storage medium, a camera, a video on demand (VoD) service provider, an over-the-top (OTT) video device, an Internet streaming service provider, a 3D video device, a virtual reality (VR) device, an augmented reality (AR) device, an image phone video device, a vehicle terminal (e.g., a vehicle (including an autonomous vehicle) terminal, an aircraft terminal, or a ship terminal) and a medical video device; and can be used to process image signals or data. For example, an OTT video device may include a game console, a Blu-ray player, an Internet-connected TV, a home theater system, a smart phone, a tablet PC, and a digital video recorder (DVR).

[0357] In addition, the processing method to which the embodiments of this document are applied may be generated in the form of a program executed by a computer and stored in a computer-readable recording medium. Multimedia data having a data structure according to the embodiments of this document may also be stored in a computer-readable recording medium. Computer-readable recording media include all kinds of storage devices and distributed storage devices in which computer-readable data is stored. Computer-readable recording media may include, for example, a Blu-ray disc (BD), a universal serial bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. Computer-readable recording media also include media embodied in the form of a carrier wave (e.g., transmission via the Internet). In addition, the bit stream generated by the encoding method may be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network.

[0358] In addition, the embodiments of this document may be embodied as a computer program product based on program codes, and the program codes may be executed on a computer according to the embodiments of this document. The program codes may be stored on a computer readable carrier.

[0359] Fig.17 represents an example of a content streaming system to which the embodiments described in this document may be applied.

[0360] refer to Fig.17 A content streaming system to which the embodiments of this document are applied may generally include an encoding server, a streaming server, a network (web) server, a media storage, a user device, and a multimedia input device.

[0361] The encoding server is used to compress the content input from the multimedia input device such as a smart phone, camera, camcorder, etc. into digital data, generate a bit stream, and transmit it to the streaming server. As another example, in the case where the multimedia input device such as a smart phone, camera, camcorder, etc. directly generates a bit stream, the encoding server can be omitted.

[0362] The bitstream may be generated by an encoding method or a bitstream generation method to which the embodiments of this document are applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0363] The streaming server sends multimedia data to the user device based on the user's request through the network server, which acts as a tool to inform the user of what services are available. When the user requests the service the user wants, the network server transfers the request to the streaming server, and the streaming server sends the multimedia data to the user. In this regard, the content streaming system may include a separate control server, and in this case, the control server is used to control the commands / responses between the various devices in the content streaming system.

[0364] The streaming server may receive content from a media storage device and / or an encoding server. For example, in the case where the content is received from the encoding server, the content may be received in real time. In this case, the streaming server may store the bitstream for a predetermined period of time to smoothly provide a streaming service.

[0365] For example, user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation, tablet PCs, tablet PCs, ultrabooks, wearable devices (e.g., watch-type terminals (smart watches), glasses-type terminals (smart glasses), head-mounted displays (HMDs)), digital TVs, desktop computers, digital signage, etc.

[0366] Each server in the content streaming system may be operated as a distributed server, and in this case, data received by each server may be processed in a distributed manner.

[0367] The claims described herein may be combined in various ways. For example, the technical features of the method claims of this specification may be combined and implemented as a device, and the technical features of the device claims of this specification may be combined and implemented as a method. In addition, the technical features of the method claims of this specification and the technical features of the device claims may be combined to be implemented as a device, and the technical features of the method claims of this specification and the technical features of the device claims may be combined and implemented as a method.

Claims

1. An image decoding method performed by a decoding device, the method comprising: Get image information from bitstream; Based on the image information, generating a reconstructed sample of the current block; as well as performing an adaptive loop filter ALF on the reconstructed samples, Wherein, executing the ALF includes: deriving filter coefficients for the ALF; and generating modified reconstructed samples based on the reconstructed samples and the filter coefficients, The image information includes an adaptation parameter set APS, Wherein, the APS includes ALF data, wherein, based on the ALF data, the filter coefficients are derived, The image information includes a slice header. The slice header includes multiple ALF-related APSID syntax elements. The slice header includes an ALF-related APSID number syntax element, wherein the number of ALF-related APSIDs is derived based on the value of the ALF-related APS ID number syntax element, and The plurality of ALF-related APSID syntax elements, which is the same number as the number of the ALF-related APS IDs, are included in the slice header.

2. The method according to claim 1, wherein: The image information includes SPS, and The SPS includes a first ALF enabling flag indicating whether the ALF is enabled.

3. The method according to claim 2, wherein: Based on the fact that the value of the first ALF enable flag is equal to 1, the slice header includes a second ALF enable flag indicating whether the ALF is enabled in the slice. 4 . The method of claim 1 , wherein the APS ID represented by one of the plurality of ALF-related APS ID syntax elements is equal to an ID of the APS including the ALF data.

5. The method according to claim 1, wherein: The slice header includes a second ALF enable flag indicating whether the ALF is enabled in the slice, wherein, based on the value of the second ALF enable flag being equal to 1, the slice header includes the ALF-related APSID number syntax element, and The value of the ALF-related APSID quantity syntax element plus 1 is equal to the ALF-related APSID quantity.

6. An image encoding method performed by an encoding device, the method comprising: Generate a reconstructed sample of the current block in the current picture; Generate adaptive loop filter ALF related information for the reconstructed sample; encode image information including information for generating the reconstructed sample and the ALF related information, The image information includes an adaptation parameter set APS, Wherein, the APS includes ALF data, wherein the ALF data includes information for deriving filter coefficients for ALF, The image information includes a slice header. The slice header includes multiple ALF-related APSID syntax elements. wherein the slice header comprises an ALF-related APSID number syntax element for signaling the number of ALF-related APSIDs, and The plurality of ALF-related APSID syntax elements, which is the same number as the number of the ALF-related APS IDs, are included in the slice header.

7. The method according to claim 6, wherein: The image information includes SPS, and The SPS includes a first ALF enabling flag indicating whether the ALF is enabled.

8. The method according to claim 7, wherein: Based on the value of the first ALF enable flag being equal to 1, the slice header includes a second ALF enable flag indicating whether the ALF is enabled in a picture or a slice.

9. The method according to claim 6, wherein: The APS ID represented by one of the plurality of ALF-related APS ID syntax elements is equal to an ID of the APS including the ALF data.

10. The method according to claim 6, wherein: The slice header includes a second ALF enable flag indicating whether the ALF is enabled in the slice, wherein, based on the value of the second ALF enable flag being equal to 1, the slice header includes the ALF-related APSID number syntax element, and The value of the ALF-related APSID quantity syntax element plus 1 is equal to the ALF-related APSID quantity.

11. A method for sending a bit stream, comprising: Generate a reconstructed sample of the current block in the current picture; Generate adaptive loop filter ALF related information for the reconstructed samples; Encoding image information to generate the bitstream, wherein the image information includes information for generating the reconstructed sample and the ALF related information; as well as sending the bit stream, The image information includes an adaptation parameter set APS, Wherein, the APS includes ALF data, wherein the ALF data includes information for deriving filter coefficients for ALF, The image information includes a slice header. The slice header includes multiple ALF-related APSID syntax elements. wherein the slice header comprises an ALF-related APSID number syntax element for signaling the number of ALF-related APSIDs, and The plurality of ALF-related APSID syntax elements, which is the same number as the number of the ALF-related APS IDs, are included in the slice header.

Images