Decoding and encoding apparatus and apparatus for transmitting data for image

Abstract

A decoding and encoding apparatus and an apparatus for transmitting data for an image are disclosed. A decoding method performed by a decoding device according to the present document comprises the steps of: parsing a flag associated with a weighted prediction from a bitstream; parsing a prediction weighting table syntax from a picture header of the bitstream based on the flag; generating a prediction sample about a current block in the current picture based on syntax elements in the prediction weighting table syntax; and generating a reconstructed sample based on the prediction sample, in which the flag includes a first flag associated with whether weighted prediction is applied to a slice of a picture parameter set of the reference bitstream and a second flag associated with whether information on the weighted prediction exists in a picture header, and the second flag can be parsed from the picture parameter set based on the first flag.

Description

[0001] This application is a divisional application of the original invention patent application with application number 202180019826.0 (international application number: PCT / KR2021 / 000467, application date: January 13, 2021, invention name: Inter-frame prediction method and device in image / video coding system). Technical Field

[0002] The present disclosure relates to a method and apparatus for performing inter-frame prediction when encoding / decoding image / video information in an image / video coding system. Background Art

[0003] Recently, the demand for high-resolution, high-quality images / videos such as 4K or 8K or above ultra-high definition (UHD) images / videos is increasing in various fields. As the image / video resolution or quality becomes higher, a relatively larger amount of information or bits is transmitted compared to conventional image / video data. Therefore, if the image / video data is transmitted via a medium such as an existing wired / wireless broadband line or is stored in a traditional storage medium, the cost for transmission and storage is easily increased.

[0004] In addition, there is growing interest and demand for virtual reality (VR) and artificial reality (AR) content and immersive media such as holograms; and there is also growing broadcasting of images / videos that exhibit image / video characteristics that are different from actual images / videos (e.g., game images / videos).

[0005] Therefore, highly efficient image / video compression technology is required to effectively compress and transmit, store, or play high-resolution, high-quality images / videos showing various characteristics as described above. Summary of the invention

[0006] Technical issues

[0007] The technical subject of this document is to provide a method and device for improving the efficiency of video / image coding.

[0008] Another technical subject of this document is to provide a method and apparatus for efficiently performing inter-frame prediction.

[0009] Yet another technical subject matter of this document is to provide a method and apparatus for efficiently signaling a prediction weighting table syntax when performing inter-frame prediction based on weighted prediction.

[0010] Still another technical subject of this document is to provide a method and apparatus for reducing signaling overhead regarding weighted prediction.

[0011] Yet another technical subject matter of this document is to provide methods and apparatus for reducing the amount of bits used for weighted prediction.

[0012] Means of solving the problem

[0013] According to an implementation of the present document, a video decoding method performed by a video decoding device may include the following steps: parsing a flag related to weighted prediction from a bitstream; parsing a prediction weighting table syntax from a picture header of the bitstream based on the flag; generating a prediction sample of a current block in a current picture based on syntax elements in the prediction weighting table syntax; and generating a reconstructed sample based on the prediction sample, wherein the flag may include a first flag related to whether the weighted prediction is applied to a slice of a picture parameter set referring to the bitstream and a second flag related to whether information about the weighted prediction is present in the picture header, and the second flag may be parsed from the picture parameter set based on the first flag.

[0014] According to another embodiment of the present document, a video encoding method performed by a video encoding device may include the following steps: generating a predicted sample of a current block by performing weighted prediction based on motion information about a current block in a current picture; generating residual information based on the predicted sample and the original sample of the current picture; and encoding image information including information about the weighted prediction and the residual information, wherein the information about the weighted prediction may include a flag related to the weighted prediction and a prediction weighting table syntax, the flag may include a first flag related to whether the weighted prediction is applied to a slice of a picture parameter set that references the image information and a second flag related to whether the information about the weighted prediction exists in a picture header of the image information, the second flag may be included in the picture parameter set based on the first flag, and the prediction weighting table syntax may be included in the picture header based on the flag.

[0015] According to another embodiment of the present document, a computer-readable digital storage medium may include information that enables a video decoding device to perform a video decoding method, and the video decoding method may include the following steps: parsing a flag related to weighted prediction from image information; parsing a prediction weighting table syntax from a picture header of the image information based on the flag; generating a prediction sample of a current block in a current picture based on syntax elements in the prediction weighting table syntax; and generating a reconstructed sample based on the prediction sample, wherein the flag may include a first flag related to whether the weighted prediction is applied to a slice of a picture parameter set that references the image information and a second flag related to whether information about the weighted prediction is present in the picture header, and the second flag may be parsed from the picture parameter set based on the first flag.

[0016] Effects of the Invention

[0017] According to the implementation of this document, the overall video / image compression efficiency can be improved.

[0018] According to the embodiments of this document, inter-frame prediction can be performed efficiently.

[0019] According to the implementation of this document, the prediction weighting table syntax can be efficiently signaled.

[0020] According to an embodiment of this document, redundant signaling for sending information about weighted prediction can be reduced.

[0021] According to the embodiments of this document, the amount of bits used for weighted prediction can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 An example of a video / image coding system to which embodiments of this document may be applied is schematically illustrated.

[0023] Figure 2 is a diagram schematically illustrating a configuration of a video / image encoding device to which an embodiment of the present document can be applied.

[0024] Figure 3 is a diagram schematically illustrating a configuration of a video / image decoding device to which an embodiment of the present document can be applied.

[0025] Figure 4 Illustrate the hierarchy of compiled images / videos.

[0026] Figure 5 An example of a video / image encoding method based on inter-frame prediction is illustrated.

[0027] Figure 6 An inter-frame predictor in an encoding device is schematically illustrated.

[0028] Figure 7 An example of a video / image decoding method based on inter-frame prediction is illustrated.

[0029] Figure 8 An inter-frame predictor in a decoding device is schematically illustrated.

[0030] Fig. 9 and Fig.10 An example of a video / image encoding method and related components according to an embodiment of this document is schematically illustrated.

[0031] Fig.11 and Fig.12 An example of a video / image decoding method and related components according to an embodiment of this document is schematically illustrated.

[0032] Fig.13An example of a content streaming system to which the embodiments disclosed in this document can be applied is illustrated. DETAILED DESCRIPTION

[0033] The disclosure of this document can be modified in various forms, and its specific embodiments will be described and illustrated in the accompanying drawings. The terms used in this document are only used to describe specific embodiments and are not intended to limit the methods disclosed in this document. Singular expressions include "at least one" expressions as long as it is clearly interpreted differently. Terms such as "including" and "having" are intended to indicate the presence of features, quantities, steps, operations, elements, components, or combinations thereof used in the document, and therefore it should be understood that the possibility of the presence or addition of one or more different features, quantities, steps, operations, elements, components, or combinations thereof is not excluded.

[0034] This document relates to video / image coding. For example, the methods / implementations disclosed in this document may be applied to methods disclosed in the Versatile Video Coding (VVC) standard. In addition, the methods / implementations disclosed in this document may be applied to methods disclosed in the Basic Video Coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the 2nd generation Audio Video Coding standard (AVS2), or the next generation video / image coding standard (e.g., H.267, H.268, etc.).

[0035] Various embodiments related to video / image coding are presented in this document, and unless otherwise specified, the above embodiments may also be performed in combination with each other.

[0036] In addition, the various configurations of the drawings described in this document are independent illustrations for explaining the functions of features that are different from each other, and do not mean that the various configurations are implemented by different hardware or different software. For example, two or more configurations can be combined to form a configuration, and a configuration can also be divided into multiple configurations. Without departing from the gist of the disclosed method of this document, embodiments of combined and / or separated configurations are included within the scope of the disclosure of this document.

[0037] In this document, the terms " / " and "," should be interpreted as indicating "and / or". For example, the expression "A / B" may mean "A and / or B". Furthermore, "A, B" may mean "A and / or B". Furthermore, "A / B / C" may mean "at least one of A, B, and / or C". Furthermore, "A / B / C" may mean "at least one of A, B, and / or C".

[0038] Furthermore, in this document, the term "or" should be interpreted as indicating "and / or". For example, the expression "A or B" may include 1) only A, 2) only B, and / or 3) both A and B. In other words, the term "or" in this document should be interpreted as indicating "additionally or alternatively".

[0039] In addition, brackets used in this document may mean "for example". Specifically, in the case of expressing "prediction (intra-frame prediction)", it may indicate that "intra-frame prediction" is proposed as an example of "prediction". In other words, the term "prediction" in this document is not limited to "intra-frame prediction", and it may indicate that "intra-frame prediction" is proposed as an example of "prediction". In addition, even in the case of expressing "prediction (i.e., intra-frame prediction)", it may indicate that "intra-frame prediction" is proposed as an example of "prediction".

[0040] In this document, technical features separately described in one drawing may be implemented separately, or may be implemented simultaneously.

[0041] Hereinafter, the embodiments of the present document will be described in detail with reference to the accompanying drawings. In addition, in all drawings, the same reference numerals may be used to indicate the same elements, and the same description of the same elements will be omitted.

[0042] Figure 1 An example of a video / image coding system to which embodiments of this document may be applied is illustrated.

[0043] Reference Figure 1 The video / image coding system may include a first device (source device) and a second device (receiving device). The source device may send the encoded video / image information or data in the form of a file or stream to the receiving device via a digital storage medium or a network.

[0044] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display, and the display may be configured as a separate device or an external component.

[0045] The video source may acquire the video / image by capturing, synthesizing or generating a video / image process. The video source may include a video / image capture device, and / or a video / image generation device. For example, the video / image capture device may include one or more cameras, a video / image archive including previously captured videos / images, etc. For example, the video / image generation device may include a computer, a tablet computer, and a smart phone, and may generate the video / image (electronically). For example, a virtual video / image may be generated by a computer, etc. In this case, the video / image capture process may be replaced by a process that generates relevant data.

[0046] The encoding device can encode the input video / image. For compression and coding efficiency, the encoding device can perform a series of processes such as prediction, transformation and quantization. The encoded data (encoded video / image information) can be output in the form of a bit stream.

[0047] The transmitter may transmit the encoded image / image information or data output in the form of a bit stream to a receiver of a receiving device in the form of a file or stream through a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter may include an element for generating a media file in a predetermined file format, and may include an element for transmission through a broadcast / communication network. The receiver may receive / extract a bit stream and transmit the received bit stream to a decoding device.

[0048] The decoding device may decode a video / image by performing a series of processes such as dequantization, inverse transformation, and prediction corresponding to the operations of the encoding device.

[0049] The renderer may render the decoded video / image. The rendered video / image may be displayed through a display.

[0050] In this document, video may refer to a series of images over time. A picture generally refers to a unit representing an image at a specific time frame, and a slice / tile refers to a unit that constitutes a part of a picture in terms of coding. A slice / tile may include one or more coding tree units (CTUs). A picture may consist of one or more slices / tiles. A picture may consist of one or more tile groups. A tile group may include one or more tiles. A tile may represent a rectangular area of ​​a CTU row within a tile in a picture. A tile may be divided into a plurality of tiles, each of which may consist of one or more CTU rows within a tile. Tiles that are not divided into a plurality of tiles may also be referred to as tiles. Tile scanning is a specific ordering of CTUs of a partitioned picture, wherein CTUs are continuously ordered in a CTU raster scan within a tile, tiles within a tile are continuously ordered in a raster scan of tiles of a tile, and tiles in a picture are continuously ordered in a raster scan of tiles of a tile. A tile is a rectangular area of ​​a CTU within a specific tile column and a specific tile row in a picture. A tile column is a rectangular area of ​​a CTU having a height equal to the height of the picture and a width specified by a syntax element in the picture parameter set. A tile row is a rectangular area of ​​a CTU having a height specified by a syntax element in the picture parameter set and a width equal to the width of the picture. Tile scanning is a specific sequential ordering of the CTUs that partition a picture, where the CTUs are ordered continuously in a CTU raster scan in tiles and the tiles in a picture are ordered continuously in a raster scan of the tiles of the picture. A slice includes an integer number of tiles of a picture that can be contained only in a single NAL unit. A slice can consist of multiple complete tiles, or only a sequence of continuous complete tiles of a tile. In this document, tile groups and slices can be used interchangeably. For example, in this document, a tile group / tile group header can be referred to as a slice / slice header.

[0051] A pixel or a picture element (pel) may refer to the smallest unit constituting a picture (or image). In addition, a "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component.

[0052] A unit may represent a basic unit of image processing. A unit may include at least one of a specific area of ​​a picture and information related to the area. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, a unit may be used interchangeably with terms such as a block or an area. In general, an M×N block may include M columns and N rows of samples (or sample arrays) or a set (or array) of transform coefficients. Alternatively, a sample may refer to a pixel value in a spatial domain, and when such a pixel value is transformed to a frequency domain, it may refer to a transform coefficient in a frequency domain.

[0053] In some cases, a unit may be used interchangeably with terms such as a block or region. In general, an M×N block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component. A sample may be used as a term corresponding to a pixel or a picture element configuring a picture (or image).

[0054] Figure 2 is a diagram schematically illustrating a configuration of a video / image encoding device to which an embodiment of this document may be applied. Hereinafter, a device referred to as a video encoding device may include an image encoding device.

[0055] Reference Figure 2 , the encoding device 200 includes and is configured with an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. According to an embodiment, the image partitioner 210, the predictor 220, the residual processor 230, the entropy encoder 240, the adder 250, and the filter 260 described above may be configured by one or more hardware components (e.g., an encoder chipset or processor). In addition, the memory 270 may include a decoded picture buffer (DPB), or may also be configured by a digital storage medium. The hardware component may also include the memory 270 as an internal / external component.

[0056] The image partitioner 210 may partition an input image (or picture, frame) input to the encoding device 200 into one or more processing units. For example, a processing unit may be referred to as a coding unit (CU). In this case, the coding unit may be recursively partitioned from a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree binary tree ternary tree (QTBTTT) structure. For example, a coding unit may be partitioned into multiple coding units with a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. In this case, for example, a quadtree structure may be applied first, and a binary tree structure and / or a ternary tree structure may be applied later. Alternatively, a binary tree structure may be applied first. The coding process according to this document may be performed based on a final coding unit that is no longer partitioned. In this case, the maximum coding unit may be directly used as the final coding unit based on coding efficiency, etc. according to image characteristics, or if necessary, the coding unit may be recursively partitioned into coding units with a deeper depth so that a coding unit with an optimal size may be used as the final coding unit. Here, the coding process may include processes such as prediction, transformation, and reconstruction (described later). In another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, each of the prediction unit and the transform unit may be split or partitioned from the above-mentioned final coding unit. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient.

[0057] The encoding device 200 may subtract a prediction signal (prediction block, prediction sample array) output from the inter predictor 221 or the intra predictor 222 from the input image signal (original block, original sample array) to generate a residual signal (residual block, residual sample array), and the generated residual signal is sent to the transformer 232. In this case, as shown, the unit that subtracts the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) in the encoding device 200 may be referred to as a subtractor 231. The predictor 220 may perform prediction on a processing target block (hereinafter, referred to as a current block) and generate a prediction block including prediction samples of the current block. The predictor 220 may determine whether intra prediction or inter prediction is applied in units of the current block or CU. As described later in the description of each prediction mode, the predictor 220 may generate various types of information (such as prediction mode information) about prediction and send the generated information to the entropy encoder 240, as described below in the description of each prediction mode. Information about the prediction may be encoded by the entropy encoder 240 and output in the form of a bitstream.

[0058] The intra-frame predictor 222 can predict the current block with reference to samples in the current picture. Depending on the prediction mode, the referenced sample can be located near the current block or can be separated from the current block. In intra-frame prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. For example, the non-directional mode can include a DC mode and a plane mode. For example, depending on the level of detail of the prediction direction, the directional mode can include 33 directional prediction modes or 65 directional prediction modes. However, this is only an example, and more or less directional prediction modes than the above number can be used according to the setting. The intra-frame predictor 222 can also use the prediction mode applied to the neighboring block to determine the prediction mode applied to the current block.

[0059] The inter-frame predictor 221 can derive the prediction block of the current block based on the reference block (reference sample array) specified by the motion vector on the reference picture. At this time, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information can be predicted in units of blocks, sub-blocks or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, bi-prediction (Bi prediction), etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same as or different from each other. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter-frame predictor 221 may configure a motion information candidate list based on the neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter predictor 221 may use motion information of a neighboring block as motion information of the current block. In the case of skip mode, unlike merge mode, a residual signal may not be sent. The motion vector prediction (MVP) mode indicates the motion vector of the current block by using the motion vector of the neighboring block as a motion vector predictor and signaling the motion vector difference.

[0060] The predictor 220 may generate a prediction signal based on various prediction methods described later. For example, the predictor 220 may apply intra prediction or inter prediction to predict a block, and may apply intra prediction and inter prediction at the same time. This may be referred to as combined inter and intra prediction (CIIP). In addition, the predictor may be based on an intra block copy (IBC) prediction mode or based on a palette mode for predicting blocks. The IBC prediction mode or the palette mode may be used for image / video coding of content such as games, for example, screen content coding (SCC). IBC basically performs prediction in the current picture, but it may be performed similarly to inter prediction in that a reference block is derived in the current picture. That is, IBC may use at least one of the inter prediction techniques described in this document. The palette mode may be considered as an example of intra coding or intra prediction. When the palette mode is applied, the sample values ​​in the picture may be signaled based on information about the palette table and the palette index.

[0061] The prediction signal generated by the predictor (including the inter-frame predictor 221 and / or the intra-frame predictor 222) may be used to generate a reconstructed signal or may be used to generate a residual signal.

[0062] The transformer 232 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, when the relationship information between pixels is illustrated as a graph, GBT means a transform obtained from a graph. CNT means a transform obtained based on a prediction signal generated by using all previously reconstructed pixels. In addition, the transform process may also be applied to square pixel blocks of the same size, or may also be applied to variable-sized blocks that are not square.

[0063] The quantizer 233 quantizes the transform coefficients and transmits the quantized transform coefficients to the entropy encoder 240, and the entropy encoder 240 encodes the quantized signal (information about the quantized transform coefficients) and outputs the encoded signal as a bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantizer 233 may rearrange the quantized transform coefficients having a block form in a one-dimensional vector form based on a coefficient scanning order, and may generate information about the transform coefficients based on the quantized transform coefficients in a one-dimensional vector form.

[0064] The entropy encoder 240 may perform various encoding methods such as, for example, exponential Golomb, context adaptive variable length coding (CAVLC), and context adaptive binary arithmetic coding (CABAC). The entropy encoder 240 may also encode information necessary for video / image reconstruction (e.g., values ​​of syntax elements, etc.) in addition to quantized transform coefficients, together or separately. The encoded information (e.g., encoded video / image information) may be transmitted or stored in units of network abstraction layer (NAL) units in the form of a bitstream. The video / image information may also include information about various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. In this document, information and / or syntax elements signaled / sent from an encoding device to a decoding device may be included in the video / image information. The video / image information may be encoded by the aforementioned encoding process and thus included in the bitstream. The bitstream may be transmitted over a network or may be stored in a digital storage medium. Here, the network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmitting unit (not shown) for transmitting a signal output from the entropy encoder 240 and / or a storage unit (not shown) for storing the signal may be configured as an internal / external element of the encoding device 200, or the transmitting unit may also be included in the entropy encoder 240.

[0065] The quantized transform coefficients output from the quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients through the dequantizer 234 and the inverse transformer 235. The adder 250 can add the reconstructed residual signal to the prediction signal output from the inter-frame predictor 221 or the intra-frame predictor 222 to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). When there is no residual for the processing target block, such as when the skip mode is applied, the prediction block can be used as a reconstructed block. The adder 250 can be referred to as a restorer or a recovery block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current picture, or can be used for inter-frame prediction of the next picture after filtering, as described below.

[0066] Additionally, luma mapping and chroma scaling (LMCS) may also be applied during the picture encoding and / or reconstruction process.

[0067] The filter 260 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 260 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and store the modified reconstructed picture in the memory 270 (specifically, the DPB of the memory 270). For example, various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filter 260 can generate various types of information related to filtering and transmit the generated information to the entropy encoder 240, as described later in the description of each filtering method. The information related to filtering can be encoded by the entropy encoder 240 and output in the form of a bit stream.

[0068] The modified reconstructed picture transmitted to the memory 270 may be used as a reference picture in the inter predictor 221. When inter prediction is applied by the encoding device, prediction mismatch between the encoding device 200 and the decoding device may be avoided and coding efficiency may be improved.

[0069] The DPB of the memory 270 may store the modified reconstructed picture for use as a reference picture in the inter-frame predictor 221. The memory 270 may store the motion information of the block from which the motion information in the current picture is derived (or encoded) and / or the motion information of the block in the reconstructed picture. The stored motion information may be transmitted to the inter-frame predictor 221 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 270 may store the reconstructed samples of the reconstructed block in the current picture and may transmit the reconstructed samples to the intra-frame predictor 222.

[0070] Figure 3 is a diagram for schematically explaining a configuration of a video / image decoding device to which an embodiment of the present document can be applied.

[0071] Reference Figure 3 , the decoding device 300 may include and be configured with an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an intra-frame predictor 331 and an inter-frame predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322. According to an embodiment, the entropy decoder 310, the residual processor 320, the predictor 330, the adder 340, and the filter 350 described above may be configured by one or more hardware components (e.g., a decoder chipset or a processor). In addition, the memory 360 may include a decoded picture buffer (DPB) or may be configured by a digital storage medium. The hardware component may also include a memory 360 as an internal / external component.

[0072] When a bit stream including video / image information is input, the decoding device 300 may respond to the Figure 2 The processing of video / image information in the illustrated encoding device reconstructs the image. For example, the decoding device 300 can derive the unit / block based on the block partition related information obtained from the bit stream. The decoding device 300 can perform decoding using a processing unit applied to the encoding device. Therefore, for example, the processing unit of decoding can be a coding unit, and the coding unit can be divided from the coding tree unit or the maximum coding unit according to a quadtree structure, a binary tree structure and / or a ternary tree structure. One or more transform units can be derived from the coding unit. In addition, the reconstructed image signal decoded and output by the decoding device 300 can be reproduced by a reproduction device.

[0073] The decoding device 300 may receive the bit stream from Figure 2The received signal can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. The decoding device may also decode the picture based on information about the parameter set and / or the general constraint information. The signaled / received information and / or syntax elements described later in this document may be decoded by a decoding process and obtained from the bitstream. For example, the entropy decoder 310 may decode the information within the bitstream based on a coding method such as exponential Golomb coding, context adaptive variable length coding (CAVLC), or context adaptive binary arithmetic coding (CABAC), and output syntax elements required for image reconstruction and quantized values ​​of transform coefficients for the residual. More specifically, the CABAC entropy decoding method can receive bins corresponding to various syntax elements in the bitstream, determine the context model by using the decoding target syntax element information, the decoding information of the decoding target block, or the information of the symbol / bin decoded in the previous stage, and perform arithmetic decoding on the bin by predicting the probability of the occurrence of the bin according to the determined context model, and generate symbols corresponding to the values ​​of the various syntax elements. In this case, the CABAC entropy decoding method can update the context model by using the information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model. The information related to the prediction among the information decoded by the entropy decoder 310 can be provided to the predictor (inter-frame predictor 332 and intra-frame predictor 331), and the residual value (i.e., quantized transform coefficients and related parameter information) that has been entropy decoded in the entropy decoder 310 can be input to the residual processor 320.

[0074] The residual processor 320 may derive a residual signal (a residual block, a residual sample, or a residual sample array). In addition, information about filtering among the information decoded by the entropy decoder 310 may be provided to the filter 350. Meanwhile, a receiver (not shown) for receiving a signal output from the encoding device may also be configured as an internal / external element of the decoding device 300, or the receiver may be a component of the entropy decoder 310. Meanwhile, the decoding device according to this document may be referred to as a video / image / picture decoding device, and the decoding device may be classified into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of the following: a dequantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.

[0075] The dequantizer 321 may dequantize the quantized transform coefficients to output the transform coefficients. The dequantizer 321 may rearrange the quantized transform coefficients in a two-dimensional block form. In this case, the rearrangement may be performed based on the coefficient scanning order performed by the encoding device. The dequantizer 321 may perform dequantization on the quantized transform coefficients using a quantization parameter (e.g., quantization step size information) and obtain the transform coefficients.

[0076] The inverse transformer 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0077] In this document, at least one of quantization / dequantization and / or transform / inverse transform may be omitted. When quantization / dequantization is omitted, the quantized transform coefficient may be referred to as a transform coefficient. When transform / inverse transform is omitted, the transform coefficient may be referred to as a coefficient or a residual coefficient or may still be referred to as a transform coefficient for consistency of expression.

[0078] In this document, the quantized transform coefficient and the transform coefficient may be referred to as a transform coefficient and a scaled transform coefficient, respectively. In this case, the residual information may include information about the transform coefficient, and the information about the transform coefficient may be signaled by residual coding syntax. The transform coefficient may be derived based on the residual information (or information about the transform coefficient), and the scaled transform coefficient may be derived by inverse transforming (scaling) the transform coefficient. The residual sample may be derived based on the inverse transform (transform) of the scaled transform coefficient. This may also be applied / expressed in other parts of this document.

[0079] The predictor 330 may perform prediction on the current block and generate a prediction block including prediction samples of the current block. The predictor may determine whether to apply intra prediction or inter prediction to the current block and determine a specific intra / inter prediction mode based on information on prediction output from the entropy decoder 310.

[0080] The predictor 330 may generate a prediction signal based on various prediction methods described below. For example, the predictor may apply intra prediction or inter prediction for predicting a block, and may apply intra prediction and inter prediction at the same time. This may be referred to as combined inter and intra prediction (CIIP). In addition, the predictor may predict a block based on an intra block copy (IBC) prediction mode or a palette mode. The IBC prediction mode or the palette mode may be used for image / video coding of content such as games, such as screen content coding (SCC). IBC may basically perform a prediction in the current picture, but may be performed similarly to inter prediction so that a reference block is derived within the current picture. That is, IBC may use at least one inter prediction technique described in this document. The palette mode may be considered as an example of intra coding or intra prediction. When the palette mode is applied, information about the palette table and the palette index may be included in the video / image information and signaled.

[0081] The intra-frame predictor 331 can predict the current block by referring to samples in the current picture. Depending on the prediction mode, the referenced sample may be located near the current block or may be separated from the current block. In intra-frame prediction, the prediction mode may include multiple non-directional modes and multiple directional modes. The intra-frame predictor 331 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

[0082] The inter-frame predictor 332 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include information about the inter-frame prediction direction (L0 prediction, L1 prediction, dual prediction, etc.). In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter-frame predictor 332 may construct a motion information candidate list based on the neighboring blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes, and information about the prediction may include information indicating the inter-frame prediction mode for the current block.

[0083] The adder 340 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the obtained residual signal to the prediction signal (prediction block or prediction sample array) output from the predictor (including the inter-frame predictor 332 and / or the intra-frame predictor 331). If there is no residual for the processing target block, such as when the skip mode is applied, the prediction block can be used as the reconstructed block.

[0084] The adder 340 may be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next block to be processed in the current picture, and as described later, may also be output through filtering or may also be used for inter prediction of the next picture.

[0085] In addition, luminance mapping and chrominance scaling (LMCS) can also be applied to the picture decoding process.

[0086] The filter 350 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 350 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 360, specifically, in the DPB of the memory 360. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0087] The (modified) reconstructed picture stored in the DPB of the memory 360 can be used as a reference picture in the inter-frame predictor 332. The memory 360 can store the motion information of the block from which the motion information in the current picture is derived (decoded) and / or the motion information of the block in the reconstructed picture. The stored motion information can be transmitted to the inter-frame predictor 332 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 360 can store the reconstructed samples of the reconstructed block in the current picture and transmit the reconstructed samples to the intra-frame predictor 331.

[0088] In this document, the embodiments described in the filter 260 , the inter predictor 221 , and the intra predictor 222 of the encoding apparatus 200 may be equally applied to or correspond to the filter 350 , the inter predictor 332 , and the intra predictor 331 .

[0089] In addition, the video / image coding method according to the present document can be performed based on the following partition structure. Specifically, the above-mentioned prediction, residual processing ((inverse) transform and (de)quantization), syntax element coding and filtering processes can be performed according to the CTU and CU (and / or TU and PU) derived based on the partition structure. The block partition process can be performed by the image partitioner 210 of the above-mentioned encoding device, and the partition related information can be processed by the entropy encoder 240 (encoding) and can be transmitted to the decoding device in the form of a bitstream. The entropy decoder 310 of the decoding device can derive the block partition structure of the current picture based on the partition related information obtained from the bitstream, and based on this, a series of processes for image decoding (e.g., prediction, residual processing, block / picture reconstruction, in-loop filtering, etc.) can be performed. The CU size and the TU size can be equal to each other, or multiple TUs can exist within the CU area. In addition, the CU size can generally represent the luminance component (sample) coding block (CB) size. The TU size can generally represent the luminance component (sample) transform block (TB) size. The chroma component (sample) CB or TB size can be derived based on the luminance component (sample) CB or TB size according to the color format (chroma format, for example, 4:4:4, 4:2:2, 4:2:0, etc.) of the picture / image according to the component ratio. The TU size can be derived based on maxTbSize. For example, if the CU size is larger than maxTbSize, multiple TUs (TBs) of maxTbSize can be derived from the CU, and transformation / inverse transformation can be performed in units of TU (TB). In addition, for example, in the case of applying intra-frame prediction, the intra-frame prediction mode / type can be derived in units of CU (or CB), and the neighboring reference sample derivation and prediction sample generation process can be performed in units of TU (or TB). In this case, one or more TUs (or TBs) may exist in one CU (or CB) area, and in this case, multiple TUs (or TBs) may share the same intra-frame prediction mode / type.

[0090] In addition, in the video / image coding according to this document, the image processing unit may have a hierarchical structure. A picture may be partitioned into one or more tiles, tiles, slices and / or tile groups. A slice may include one or more tiles. A tile may include one or more CTU rows within a tile. A slice may include an integer number of tiles of a picture. A tile group may include one or more tiles. A tile may include one or more CTUs. A CTU may be partitioned into one or more CUs. A tile represents a rectangular area of ​​a CTU within a specific tile column and a specific tile row in a picture. A tile group may include an integer number of tiles according to a tile raster scan in a picture. A slice header may carry information / parameters that can be applied to a corresponding slice (a block in a slice). If the encoding / decoding device has a multi-core processor, the encoding / decoding process for tiles, slices, tiles and / or tile groups may be processed in parallel. In this document, slices or tile groups may be used interchangeably. That is, a tile group header may be referred to as a slice header. Here, the slice may have one of slice types including an intra (I) slice, a predicted (P) slice, and a bi-predicted (B) slice. When predicting a block in an I slice, inter prediction may not be used, and only intra prediction may be used. Of course, even in this case, signaling may be performed by coding the original sample value without prediction. With respect to a block in a P slice, intra prediction or inter prediction may be used, and in the case of using inter prediction, only unidirectional prediction may be used. In addition, with respect to a block in a B slice, intra prediction or inter prediction may be used, and in the case of using inter prediction, up to bi-prediction may be used to the maximum.

[0091] Taking into account coding efficiency or parallel processing or according to the characteristics of the video image (e.g., resolution), the encoding device can determine the tile / tile group, tile, slice, and maximum and minimum coding unit sizes, and their information or information that can derive them can be included in the bitstream.

[0092] The decoding device can obtain information indicating the tiles / tile groups, tiles, and slices of the current picture, and whether the CTU in the tiles has been partitioned into multiple coding units. By acquiring (sending) such information only under certain conditions, efficiency can be improved.

[0093] As described above, a picture may include multiple slices, and a slice may include a slice header and slice data. In this case, a picture header may be further added for multiple slices in a picture (a set of slice headers and slice data). The picture header (picture header syntax) may include information / parameters commonly applied to the picture. The slice header (slice header syntax) may include information / parameters commonly applied to the slice. An adaptive parameter set (APS) or a picture parameter set (PPS) may include information / parameters commonly applied to one or more pictures. A sequence parameter set (SPS) may include information / parameters commonly applied to one or more sequences. A video parameter set (VPS) may include information / parameters commonly applied to multiple layers. A decoding parameter set (DPS) may include information / parameters commonly applied to the overall video. The DPS may include information / parameters related to the cascade of a coded video sequence (CVS).

[0094] In this document, the high-level syntax may include at least one of an APS syntax, a PPS syntax, an SPS syntax, a VPS syntax, a DPS syntax, a picture header syntax, and a slice header syntax.

[0095] In addition, for example, information on partitioning and configuration of tiles / tile groups / tiles / slices may be configured in an encoding device based on a high-level syntax and may be transmitted to a decoding device in the form of a bitstream.

[0096] Figure 4 The hierarchical structure of compiled images / videos is shown exemplarily.

[0097] Reference Figure 4 The compiled image / video is divided into the VCL (Video Coding Layer) that handles the image / video decoding process and itself, the subsystem that sends and stores the compiled information, and the Network Abstraction Layer (NAL) that exists between the VCL and the subsystem and is responsible for the network adaptation function.

[0098] The VCL may generate VCL data including compressed image data (slice data), or a parameter set including a picture parameter set (PPS), a sequence parameter set (SPS), a video parameter set (VPS), etc., or a supplemental enhancement information (SEI) message additionally required for image decoding processing.

[0099] In NAL, a NAL unit may be generated by adding header information (NAL unit header) to a raw byte sequence payload (RBSP) generated in a VCL. In this case, RBSP refers to slice data, parameter sets, SEI messages, etc. generated in a VCL. The NAL unit header may include NAL unit type information specified according to the RBSP data included in the corresponding NAL unit.

[0100] As shown in the figure, the NAL unit can be divided into a VCL NAL unit and a non-VCL NAL unit according to the RBSP generated in the VCL. The VCL NAL unit may refer to a NAL unit including information about the image (slice data), and the non-VCL NAL unit may refer to a NAL unit containing information required for decoding the image (parameter set or SEI message).

[0101] By attaching header information according to the data standard of the subsystem, the above-mentioned VCL NAL unit and non-VCL NAL unit can be sent through the network. For example, the NAL unit can be transformed into a predetermined standard data form such as H.266 / VVC file format, real-time transport protocol (RTP), transport stream (TS), etc., and sent through various networks.

[0102] As described above, in a NAL unit, a NAL unit type may be specified according to an RBSP data structure included in a corresponding NAL unit, and information on the NAL unit type may be stored in a NAL unit header and signaled.

[0103] For example, NAL units can be roughly classified into VCL NAL unit types and non-VCL NAL unit types according to whether the NAL unit includes information about the image (slice data). VCL NAL unit types can be classified according to the characteristics and types of pictures included in the VCL NAL unit, and non-VCL NAL unit types can be classified according to the types of parameter sets.

[0104] The following are examples of NAL unit types specified according to the type of parameter sets included by the non-VCL NAL unit type.

[0105] -APS (Adaptation Parameter Set) NAL unit: the type of NAL unit that includes APS

[0106] -DPS (Decoding Parameter Set) NAL unit: the type of NAL unit that includes the DPS

[0107] -VPS (Video Parameter Set) NAL unit: the type of NAL unit that contains the VPS

[0108] -SPS (Sequence Parameter Set) NAL unit: the type of NAL unit that includes SPS

[0109] -PPS (Picture Parameter Set) NAL unit: the type of NAL unit that includes PPS

[0110] - PH (Picture Header) NAL unit: the type of NAL unit including PH

[0111] The above-mentioned NAL unit type has syntax information of the NAL unit type, and the syntax information may be stored in the NAL unit header and signaled. For example, the syntax information may be nal_unit_type, and the NAL unit type may be specified by the nal_unit_type value.

[0112] In addition, as described above, a picture may include multiple slices, and a slice may include a slice header and slice data. In this case, a picture header may be further added to multiple slices (slice headers and slice data sets) in a picture. The picture header (picture header syntax) may include information / parameters that are commonly applicable to the picture. The slice header (slice header syntax) may include information / parameters that are commonly applicable to the slice. APS (APS syntax) or PPS (PPS syntax) may include information / parameters that are commonly applicable to one or more slices or pictures. SPS (SPS syntax) may include information / parameters that are commonly applicable to one or more sequences. VPS (VPS syntax) may include information / parameters that are commonly applicable to multiple layers. DPS (DPS syntax) may include information / parameters that are commonly applicable to the entire video. DPS may include information / parameters related to the cascade of a coded video sequence (CVS).

[0113] In this document, the high level syntax (HLS) may include at least one of an APS syntax, a PPS syntax, an SPS syntax, a VPS syntax, a DPS syntax, a picture header syntax, and a slice header syntax.

[0114] In addition, for example, information on partitioning and configuration of tiles / tile groups / tiles / slices may be configured in an encoding terminal based on a high-level syntax and may be transmitted to a decoding device in the form of a bitstream.

[0115] In this document, the image / video encoded in the encoding device and notified to the decoding device in the form of a bitstream may include not only information related to intra-picture partitions, intra-frame / inter-frame prediction information, residual information, and in-loop filtering information, but also information included in a slice header, information included in a picture header, information included in an APS, information included in a PPS, information included in an SPS, information included in a VPS, and / or information included in a DPS. In addition, the image / video information may also include information of a NAL unit header.

[0116] The video / image encoding process based on inter-frame prediction may schematically include, for example, the following contents.

[0117] Figure 5 An example of a video / image encoding method based on inter-frame prediction is illustrated, Figure 6 An inter-frame predictor in an encoding device is schematically illustrated.

[0118] Reference Figure 5 and Figure 6 , the encoding device performs inter prediction on the current block (S500). The encoding device may derive an inter prediction mode and motion information for the current block, and may generate a prediction sample of the current block. Here, the processes for determining the inter prediction mode, deriving motion information, and generating prediction samples may be performed simultaneously, or one process may be performed before another process. For example, the inter predictor 221 of the encoding device may include a prediction mode determiner 221_1, a motion information deriver 221_2, and a prediction sample deriver 221_3, wherein the prediction mode determiner 221_1 may determine the prediction mode of the current block, the motion information deriver 221_2 may derive motion information about the current block, and the prediction sample deriver 221_3 may derive the prediction sample of the current block. For example, the inter predictor of the encoding device may search for a block similar to the current block in a predetermined area (search area) of a reference picture through motion estimation, and may derive a reference block whose difference with the current block is a minimum value or a predetermined reference level or less. The inter-frame predictor can derive a reference picture index based on the reference block, which indicates the reference picture where the reference block is located, and can derive a motion vector based on the position difference between the reference block and the current block. The encoding device can determine the mode to be applied to the current block among various prediction modes. The encoding device can compare the rate-distortion (RD) cost of various prediction modes and can determine the best prediction mode for the current block.

[0119] For example, when a skip mode or a merge mode is applied to a current block, the encoding device may construct a merge candidate list, and may derive a reference block among reference blocks indicated by a merge candidate included in the merge candidate list, the difference between the reference block and the current block being a minimum value or a predetermined reference level or less. In this case, a merge candidate associated with the derived reference block may be selected, and merge index information indicating the selected merge candidate may be generated and signaled to the decoding device. Motion information about the current block may be derived using motion information about the selected merge candidate.

[0120] In another example, when the (A)MVP mode is applied to the current block, the encoding device may construct an (A)MVP candidate list, and may use the motion vector of the MVP candidate selected from the motion vector predictor (MVP) candidates included in the (A)MVP candidate list as the MVP of the current block. For example, in this case, the motion vector indicating the reference block derived by motion estimation may be used as the motion vector of the current block, and the MVP candidate having the motion vector with the smallest difference from the motion vector of the current block among the MVP candidates may be the selected MVP candidate. A motion vector difference (MVD) may be derived, which is the difference obtained by subtracting the MVP from the motion vector of the current block. In this case, information about the MVD may be notified to the decoding device with a signal. When the (A)MVP mode is applied, the value of the reference picture index may be configured as reference picture index information, and may be separately notified to the decoding device with a signal.

[0121] The encoding apparatus may induce residual samples based on the prediction samples (S510). The encoding apparatus may induce residual samples by comparing original samples of the current block with the prediction samples.

[0122] The encoding device encodes the image information including prediction information and residual information (S520). The encoding device may output the encoded image information in the form of a bitstream. The prediction information is information related to the prediction process, and may include prediction mode information (e.g., a skip flag, a merge flag, or a mode index) and information about motion information. The information about the motion information may include candidate selection information (e.g., a merge index, an mvp flag, or an mvp index), which is information for deriving a motion vector. In addition, the information about the motion information may include information about the MVD and / or reference picture index information. In addition, the information about the motion information may include information indicating whether L0 prediction, L1 prediction, or dual prediction is applied. The residual information is information about the residual sample. The residual information may include information about the quantized transform coefficients of the residual sample.

[0123] The output bitstream may be stored in a (digital) storage medium and sent to a decoding device, or may be sent to a decoding device over a network.

[0124] As described above, the encoding device can generate a reconstructed picture (including a reconstructed sample and a reconstructed block) based on the reference sample and the residual sample, and the reconstructed picture is used for the encoding device to derive the same result as the prediction result derived by the decoding device, and is used to improve the coding efficiency. Therefore, the encoding device can store the reconstructed picture (or the reconstructed sample and the reconstructed block) in the memory, and can use it as a reference picture for inter-frame prediction. As described above, the in-loop filtering process can be further applied to the reconstructed picture.

[0125] For example, the video / image decoding process based on inter-frame prediction may schematically include the following contents.

[0126] Figure 7 An example of a video / image decoding method based on inter-frame prediction is illustrated, and Figure 8 The inter-frame predictor in a decoding device is schematically illustrated.

[0127] The decoding device may perform operations corresponding to the above operations performed by the encoding device. The decoding device may predict the current block based on the received prediction information and may derive a prediction sample.

[0128] Specifically, refer to Figure 7 and Figure 8 The decoding device may determine a prediction mode of the current block based on the prediction information received from the bitstream (S700). The decoding device may determine which inter-frame prediction mode to apply to the current block based on the prediction mode information in the prediction information.

[0129] For example, whether to apply the merge mode to the current block or whether to determine the (A) MVP mode may be determined based on the merge flag. Alternatively, one of various inter prediction mode candidates may be selected based on the merge index. The inter prediction mode candidate may include various inter prediction modes, such as skip mode, merge mode, and / or (A) MVP mode.

[0130] The decoding device derives motion information about the current block based on the determined inter-frame prediction mode (S710). For example, when the skip mode or merge mode is applied to the current block, the decoding device can construct a merge candidate list described later, and can select a merge candidate from the merge candidates included in the merge candidate list. The selection can be performed based on the above-mentioned selection information (merge index). The motion information about the selected merge candidate can be used to derive the motion information about the current block. The motion information about the selected merge candidate can be used as the motion information about the current block.

[0131] In another example, when the (A)MVP mode is applied to the current block, the decoding device may construct an (A)MVP candidate list, and may use the motion vector of a motion vector predictor (MVP) candidate selected from the MVP candidates included in the (A)MVP candidate list as the MVP of the current block. The selection may be performed based on the above-mentioned selection information (MVP flag or MVP index). In this case, the decoding device may derive the MVD of the current block based on the information about the MVD, and may derive the motion vector of the current block based on the MVP and MVD of the current block. In addition, the decoding device may derive the reference picture index of the current block based on the reference picture index information. The picture indicated by the reference picture index in the reference picture list of the current block may be derived as the reference picture referenced by the inter-frame prediction of the current block.

[0132] The motion information about the current block may be derived without constructing a candidate list, in which case the construction of the candidate list described above may be omitted.

[0133] The decoding device may generate a prediction sample of the current block based on the motion information about the current block (S720). In this case, a reference picture may be derived based on a reference picture index of the current block, and a sample of the reference block indicated by a motion vector of the current block in the reference picture may be used to derive the prediction sample of the current block. In this case, as described later, a prediction sample filtering process of all or some prediction samples of the current block may be further performed according to circumstances.

[0134] For example, the inter-frame predictor 332 of the decoding device may include a prediction mode determiner 332_1, a motion information deriver 332_2 and a prediction sample deriver 332_3, wherein the prediction mode determiner 332_1 can determine the prediction mode of the current block based on the received prediction mode information, the motion information deriver 332_2 can derive motion information (motion vector and / or reference picture index) about the current block based on the received information about the motion information, and the prediction sample deriver 332_3 can derive the prediction sample of the current block.

[0135] The decoding device generates a residual sample of the current block based on the received residual information (S730). The decoding device can generate a reconstructed sample of the current block based on the predicted sample and the residual sample, and can generate a reconstructed picture based on the reconstructed sample (S740). Subsequently, as described above, the in-loop filtering process can be further applied to the reconstructed picture.

[0136] The prediction block of the current block may be derived based on motion information derived according to the prediction mode of the current block. The prediction block may include prediction samples (prediction sample array) of the current block. When the motion vector of the current block indicates a partial sample unit, an interpolation process may be performed, by which the prediction samples of the current block may be derived based on reference samples in the partial sample unit in the reference picture. When affine inter prediction is applied to the current block, prediction samples may be generated based on motion vectors (MVs) in sample / subblock units. When bi-prediction is applied, prediction samples derived by weighted sum or weighted average of prediction samples derived based on L0 prediction (i.e., prediction using reference pictures in reference picture list L0 and MVL0) and prediction samples derived based on L1 prediction (i.e., prediction using reference pictures in reference picture list L1 and MVL1 (according to the stage)) may be used as prediction samples of the current block. The case where bi-prediction is applied and the reference picture for L0 prediction and the reference picture for L1 prediction are located in different time directions relative to the current picture (i.e., corresponding to bi-prediction and bi-directional prediction) is called true bi-prediction.

[0137] As described above, reconstructed samples and reconstructed pictures may be generated based on the derived prediction samples, and then an in-loop filtering process may be performed.

[0138] In inter-frame prediction, weighted sample prediction can be used. Weighted sample prediction can be called weighted prediction. Weighted prediction can be applied when the slice type of the current slice where the current block (e.g., CU) is located is a P slice or a B slice. That is, weighted prediction can be used not only when dual prediction is applied, but also when single prediction is applied. For example, as described below, weighted prediction can be determined based on weightedPredFlag, and the value of weightedPredFlag can be determined based on the signaled pps_weighted_pred_flag (in the case of a P slice) or pps_weighted_bipred_flag (in the case of a B slice). For example, when slice_type is p, weightedPredFlag can be set to pps_weighted_pred_flag. Otherwise (when slice_type is B), weightedPredFlag can be set to pps_weighted_bipred_flag.

[0139] The prediction samples or values ​​of the prediction samples as the output of the weighted prediction may be referred to as pbSamples.

[0140] The weighted prediction process can be largely divided into the default weighted (sample) prediction process and the explicit weighted (sample) prediction process. The weighted (sample) prediction process can refer only to the explicit weighted (sample) prediction process. For example, when the value of weightedPredFlag is 0, the value of the prediction sample (pbSamples) can be derived based on the default weighted (sample) prediction process. When the value of weightedPredFlag is 1, the value of the prediction sample (pbSamples) can be derived based on the explicit weighted (sample) prediction process.

[0141] When bi-prediction is applied to the current block, the prediction sample can be derived based on weighted averaging. Generally, the bi-prediction signal (i.e., bi-prediction sample) can be derived by simply averaging the L0 prediction signal (L0 prediction sample) and the L1 prediction signal (L1 prediction sample). That is, the bi-prediction sample is derived as the average of the L0 prediction sample based on the L0 reference picture and MVL0 and the L1 prediction sample based on the L1 reference picture and MVL1. However, according to this document, when bi-prediction is applied, the bi-prediction signal (bi-prediction sample) can be derived by weighted averaging the L0 prediction signal and the L1 prediction signal.

[0142] Bidirectional optical flow (BDOF) can be used to refine the bi-prediction signal. BDOF is used to generate prediction samples by calculating improved motion information when bi-prediction is applied to the current block (eg, CU), and the process of calculating the improved motion information can be included in the motion information derivation operation.

[0143] For example, BDOF can be applied in 4×4 sub-block levels. That is, BDOF can be performed in units of 4×4 sub-blocks in the current block. BDOF can only be applied to the luma component. Alternatively, BDOF can only be applied to the chroma component, or can also be applied to the luma component and the chroma component.

[0144] As described above, a high level syntax (HLS) may be compiled for video / image coding / signaling. Video / image information may be included in the HLS.

[0145] A coded picture may include one or more slices. Parameters describing the coded picture are signaled in the picture header, and parameters describing the slice are signaled in the slice header. The picture header is carried in the form of an independent NAL unit. The slice header is present at the beginning of the NAL unit, which includes the payload of the slice (i.e., the slice data).

[0146] Each picture is associated with a picture header. A picture may include different types of slices (intra-coded slices (i.e., I slices) and inter-coded slices (i.e., P slices and B slices). Therefore, the picture header may include syntax elements required for intra-frame slices and inter-frame slices of the picture.

[0147] A picture may be divided into sub-pictures, tiles and / or slices. Sub-picture signaling may be present in the sequence parameter set (SPS), and tile and square slice signaling may be present in the picture parameter set (PPS). Raster scan slice signaling may be present in the slice header.

[0148] When weighted prediction is applied to inter prediction of the current block, weighted prediction may be performed based on information about weighted prediction.

[0149] The weighted prediction process can be started based on two flags in SPS.

[0150] For example, the syntax elements shown in Table 1 below may be included in the SPS syntax regarding weighted prediction.

[0151] [Table 1]

[0152]

[0153] In Table 1, a value of sps_weighted_pred_flag equal to 1 may indicate that weighted prediction is applied to the P slice referencing the SPS.

[0154] A value of sps_weighted_bipred_flag equal to 1 may indicate that weighted prediction is applied to B slices referencing the SPS.

[0155] A value of sps_weighted_bipred_flag equal to 0 may indicate that weighted prediction is not applied to B slices referencing the SPS.

[0156] Two flags signaled in the SPS indicate whether weighted prediction is applied to P slices and B slices in a coded video sequence (CVS).

[0157] The syntax elements shown in Table 2 below may be included in the PPS syntax regarding weighted prediction.

[0158] [Table 2]

[0159]

[0160] In Table 2, a value of pps_weighted_pred_flag equal to 0 may indicate that weighted prediction is not applied to the P slice referencing the PPS. A value of pps_weighted_pred_flag equal to 1 may indicate that weighted prediction is applied to the P slice referencing the PPS. When the value of sps_weighted_pred_flag is 0, the value of pps_weighted_pred_flag is 0.

[0161] A value of pps_weighted_bipred_flag equal to 0 may indicate that weighted prediction is not applied to B slices referencing the PPS. A value of pps_weighted_bipred_flag equal to 1 may indicate that explicit weighted prediction is applied to B slices referencing the PPS. When the value of sps_weighted_bipred_flag is 0, the value of pps_weighted_bipred_flag is 0.

[0162] In addition, the syntax elements shown in Table 3 below may be included in the slice header syntax.

[0163] [Table 3]

[0164]

[0165]

[0166] In Table 3, slice_pic_parameter_set_id indicates a value of pps_pic_parameter_set_id of a used PPS. The value of slice_pic_parameter_set_id is within the range from 0 to 63, included.

[0167] The value of the temporary ID (TempralID) of the current picture needs to be greater than or equal to the TempralID value of the PPS having the same pps_pic_parameter_set_id as slice_pic_parameter_set_id.

[0168] The prediction weighting table syntax may include information about the weighted predictions shown in Table 4 below.

[0169] [Table 4]

[0170]

[0171]

[0172] In Table 4, luma_log2_weight_denom is the base-2 logarithm of the denominator of all luma weighting factors. The value of luma_log2_weight_denom is within the range from 0 to 7, inclusive.

[0173] delta_chroma_log2_weight_denom is the difference of the base-2 logarithms of the denominators of all chroma weighting factors. When delta_chroma_log2_weight_denom is not present, delta_chroma_log2_weight_denom is inferred to be 0.

[0174] ChromaLog2WeightDenom is derived as luma_log2_weight_denom+delta_chroma_log2_weight_denom, and its value is in the range from 0 to 7, included.

[0175] A value of luma_weight_10_flag[i] equal to 1 indicates that weighting factors for luma components predicted using list 0 (L0) of (reference pictures) RefPicList[0][i] are present. A value of luma_weight_10_flag[i] equal to 0 indicates that these weighting factors are not present.

[0176] A value of chroma_weight_10_flag[i] equal to 1 indicates that weighting factors for the chroma prediction values ​​of the L0 prediction of RefPicList[0][i] are used. A value of chroma_weight_10_flag[i] equal to 0 indicates that these weighting factors are not present.

[0177] When chroma_weight_10_flag[i] is not present, chroma_weight_10_flag[i] is inferred to be 0.

[0178] delta_luma_weight_10[i] is the difference in weighting factors applied to the luma prediction values ​​of the L0 prediction using RefPicList[0][i].

[0179] LumaWeightL0[i] is inferred as (1 << luma_log2_weight_denom) + delta_luma_weight_l0[i]. When luma_weight_10_flag[i] is 1, the value of delta_luma_weight_10[i] is included in the range from -128 to 127. When luma_weight_10_flag[i] is 0, LumaWeightL0[i] is inferred as 2 luma _ log2 _ weight _ denom 。

[0180] luma_offset_10[i] is the cumulative offset applied to the luma prediction value for L0 prediction using RefPicList[0][i]. The value of luma_offset_10[i] is included in the range from -128 to 127. When the value of luma_weight_10_flag[i] is 0, the value of luma_offset_10[i] is inferred as 0.

[0181] delta_chroma_weight_l0[i][j] is the difference in the weighting factor for the chroma prediction value for L0 prediction using RefPicList[0][i], where j is 0 for Cb and j is 1 for Cr.

[0182] ChromaWeightL0[i][j] is derived as (1 << Chromalog2WeightDenom) + delta_chroma_weight_l0[i][j]. When chroma_weight_10_flag[i] is 1, the value of delta_chroma_weight_10[i][j] is included in the range from -128 to 127. When chroma_weight_l0_flag[i] is 0, ChromaWeightL0[i][j] is inferred as 2 ChromaLog2WeightDenom 。

[0183] delta_chroma_offset_l0[i][j] is the cumulative offset applied to the chroma prediction value for L0 prediction using RefPicList[0][i], where j is 0 for Cb and j is 1 for Cr.

[0184] The value of delta_chroma_offset_10[i][j] is included in the range from -4x128 to 4x127. When the value of chroma_weight_10_flag[i] is 0, the value of ChromaOffsetL0[i][j] is inferred to be 0.

[0185] The prediction weighting table syntax is often used to modify sequences when the scene changes. When the PPS flag for weighted prediction is enabled and the slice type is P or when the PPS flag for weighted bi-prediction is enabled and the slice type is B, the existing prediction weighting table syntax is signaled in the slice header. However, it is often the case that when the scene changes, the prediction weighting table needs to be adjusted for one or more frames. Typically, when multiple frames share a PPS, it may not be necessary to signal information about the weighted predictions for all frames that reference the PPS.

[0186] The following drawings are provided to describe specific examples of this document. Since specific terms of the devices illustrated in the drawings or specific signal / message terms are provided for illustration, the technical features of the present disclosure are not limited to the specific terms used in the following drawings.

[0187] This document provides the following methods to solve the above problems. These methods can be applied independently or in combination with each other.

[0188] 1. Tools that can apply weighted prediction at picture level instead of slice level (information about weighted prediction). Weighted values ​​are applied to a specific reference picture of a picture and are used for all slices of the picture.

[0189] a. Alternatively, the tools for weighted prediction may be applied at the picture level or the slice level, but may not be applied at both levels for the same picture. That is, when weighted prediction is applied at the picture level, weighted prediction does not require additional signaling for each slice of the picture.

[0190] b. Alternatively, tools for weighted prediction can be applied at both picture level and slice level.

[0191] 2. The prediction weighting table syntax can be signaled at the picture level instead of the slice level. To this end, information about the weighted prediction syntax can be signaled in the picture header (PH) or the picture parameter set (PPS).

[0192] 3. Alternatively, when the tool for weighted prediction is applicable to a picture level or a slice level, the tool for weighted prediction may be signaled at the picture level or the slice level.

[0193] a. Picture level signaling exists in the picture header.

[0194] b. Slice level signaling exists in the slice header.

[0195] 4. Alternatively, when the tool for weighted prediction is applicable to the picture level or the slice level, the tool for weighted prediction may be signaled at the picture level and then may be overridden at the slice level.

[0196] a. Picture level signaling exists in the picture header.

[0197] b. Slice level signaling exists in the slice header.

[0198] 5. When the prediction weighting table is present in the picture header, the number of weighted reference pictures can be signaled explicitly. One number for reference picture list 0 and one number for reference picture list 1 can be signaled. When the prediction weighting table is present in the slice header, the number can be inferred from the number of active reference pictures for reference picture list 0 and reference picture list 1.

[0199] 6. When the tools for weighted prediction (eg, prediction weighting table) are signaled in the picture header, reference picture signaling for a picture (and slices of a picture) may be present in the picture header.

[0200] 7. When weighted prediction is applied at the picture level, if weighted prediction is applied to a picture, all slices of the picture can have the same active reference pictures. This includes the order of active reference pictures in the reference picture lists (i.e., L0 for P slices, L0 and L1 for B slices).

[0201] 8. Alternatively, when the above does not apply, the following may apply.

[0202] a. The signaling of weighted prediction is independent of the signaling of reference picture lists. That is, in the signaling of the prediction weighting table, there is no assumption about the order of reference pictures in the RPL.

[0203] b. There is no signaling of weighted prediction values ​​for reference pictures in L0 and L1. For reference pictures, the weighted values ​​are provided directly.

[0204] c. Use only one loop, instead of two, to signal the weight values ​​of reference pictures. In each loop, identify the reference picture associated with the weight value that is signaled first.

[0205] d. Reference picture identification is based on the picture order count (POC) value.

[0206] e. For bit saving, the delta POC value between the reference picture and the current picture may be signaled instead of signaling the POC value of the reference picture.

[0207] 9. In addition to item 4, in order to signal the delta POC value between the reference picture and the current picture, the following may be further specified so that the absolute delta POC value may be signaled as follows.

[0208] a. The first signaled deltaPOC is the delta between the reference picture's POC and the current picture's POC.

[0209] b. The remaining signaled delta POCs (ie, the case where i starts from 1) are the deltas between the POC of the i-th reference picture and the POC of the (i-1)-th reference picture.

[0210] 10. The two flags in PPS can be unified into a single control flag (eg, pps_weighted_pred_flag). This flag can be used to indicate the presence of additional flags in the picture header.

[0211] a. The flag in PH can be conditional on the PPS flag and can further indicate the presence of pred_weighted_table() data (prediction weighting table syntax) when the NAL unit type is not instantaneous decoding refresh (IDR).

[0212] 11. The two flags signaled in PPS (pps_weighted_pred_flag and pps_weighted_bipred_flag) can be unified into one flag. The one flag can use the existing name of pps_weighted_pred_flag.

[0213] 12. A flag may be signaled in the picture header to indicate whether weighted prediction is applied to the picture associated with the picture header. The flag may be called pic_weighted_pred_flag.

[0214] a. The presence of pic_weighted_pred_flag may be conditional on the value of pps_weighted_pred_flag. When the value of pps_weighted_pred_flag is 0, pic_weighted_pred_flag is not present and its value may be inferred to be 0.

[0215] b. When the value of pic_weighted_pred_flag is 1, the signaling of pred_weighted_table() may exist in the picture header.

[0216] 13. Alternatively, when weighted prediction is enabled (ie, the value of pps_weighted_pred_flag is 1 or the value of pps_weighted_bipred_flag is 1), information about weighted prediction may still exist in the slice header, and the following may apply.

[0217] a. A new flag may be signaled to indicate whether information about weighted prediction is present in the slice header. The flag may be called slice_weighted_pred_present_flag.

[0218] b. The presence of slice_weighted_pred_present_flag may be determined according to the slice type and the values ​​of pps_weighted_pred_flag and pps_weighted_bipred_flag.

[0219] In this document, information about weighted prediction may include information / syntax elements related to weighted prediction described in Tables 1 to 4. Video / image information may include various inter-frame prediction information, such as information about weighted prediction, residual information, and inter-frame prediction mode information. Inter-frame prediction mode information may include information / syntax elements, such as information indicating whether merge mode or MVP mode is applied to the current block, and selection information for selecting one of the motion candidates in the motion candidate list. For example, when merge mode is applied to the current block, a merge candidate list is constructed based on neighboring blocks of the current block, and one candidate for deriving motion information about the current block can be selected / used from the merge candidate list (based on the merge index). In another example, when the MVP mode is applied to the current block, an MVP candidate list can be constructed based on neighboring blocks of the current block, and one candidate for deriving motion information about the current block can be selected / used from the MVP candidate list (based on the MVP flag).

[0220] In one embodiment, for weighted prediction in inter prediction, the PPS may include the syntax elements shown in Table 5 below, and the semantics of the syntax elements may be as shown in Table 6 below.

[0221] [Table 5]

[0222]

[0223] [Table 6]

[0224]

[0225] Referring to Table 5 and Table 6, a value of pps_weighted_pred_flag equal to 0 may indicate that weighted prediction is not applied to a P or B slice referencing a PPS. A value of pps_weighted_pred_flag equal to 1 may indicate that weighted prediction is applied to a P or B slice referencing a PPS.

[0226] In addition, the picture header may include the syntax elements shown in Table 7 below, and the semantics of the syntax elements may be as shown in Table 8 below.

[0227] [Table 7]

[0228]

[0229] [Table 8]

[0230]

[0231] Referring to Tables 7 and 8, a value of pic_weighted_pred_flag equal to 0 may indicate that weighted prediction is not applied to a P or B slice of a reference picture header. A value of pic_weighted_pred_flag equal to 1 may indicate that weighted prediction is applied to a P or B slice of a reference picture header.

[0232] When the value of pic_weighted_pred_flag is 1, all slices associated with the picture header in the picture may have the same reference picture list. Otherwise, when the value of pic_weighted_pred_flag is 1, the value of pic_rpl_present_flag may be 1.

[0233] In the absence of the above conditions, pic_weighted_pred_flag may be signaled as shown in Table 9 below.

[0234] [Table 9]

[0235]

[0236] The slice header may include the syntax elements shown in Table 10 below.

[0237] [Table 10]

[0238]

[0239] The prediction weighting table syntax may include the syntax elements shown in Table 11 below, and the semantics of the syntax elements may be as shown in Table 12 below.

[0240] [Table 11]

[0241]

[0242] [Table 12]

[0243]

[0244] Referring to Table 11 and Table 12, num_10_weighted_ref_pics may indicate the number of weighted reference pictures in reference picture list 0. The value of num_10_weighted_ref_pics is within the range from 0 to MaxDecPicBuffMinus1+14, included.

[0245] num_11_weighted_ref_pics may indicate the number of weighted reference pictures in reference picture list 1. The value of num_11_weighted_ref_pics is within the range from 0 to MaxDecPicBuffMinus1+14, included.

[0246] A value of luma_weight_10_flag[i] equal to 1 indicates the presence of a weighting factor for the luma component predicted using List 0 (L0) of RefPicList[0][i].

[0247] A value of chroma_weight_10_flag[i] equal to 1 indicates that weighting factors for chroma prediction values ​​using L0 prediction of RefPicList[0][i] are present. A value of chroma_weight_10_flag[i] equal to 0 indicates that these weighting factors are not present.

[0248] A value of luma_weight_11_flag[i] equal to 1 indicates the presence of a weighting factor for the luma component predicted using List 1 (L1) of RefPicList[0][i].

[0249] chroma_weight_11_flag[i] indicates the presence of weighting factors for chroma prediction values ​​using L1 prediction of RefPicList[0][i]. A value of chroma_weight_10_flag[i] equal to 0 indicates that these weighting factors are not present.

[0250] For example, when weighted prediction is applied to the current block, the encoding device may generate quantity information about weighted reference pictures in the reference picture list of the current block based on the weighted prediction. The quantity information may refer to quantity information about the weights signaled for the items (reference pictures) in the L0 reference picture list and / or the L1 reference picture list. That is, the value of the quantity information may be equal to the number of weighted reference pictures in the reference picture list (L0 and / or L1). Therefore, when the value of the quantity information is n, the prediction weighting table syntax may include n weighting factor related flags for the reference picture list. The weighting factor related flag may correspond to luma_weight_l0_flag, luma_weight_l1_flag, chroma_weight_l0_flag and / or chroma_weight_l0_flag of Table 11. The weight of the current picture may be derived based on the weighting factor related flag.

[0251] When weighted bi-prediction is applied to the current block, the prediction weighting table syntax may independently include information about the number of weighted reference pictures in the L1 reference picture list and information about the number of weighted reference pictures in the L0 reference picture list, as shown in Table 11. A weighting factor related flag may be independently included for each of the information about the number of weighted reference pictures in the L1 reference picture list and the information about the number of weighted reference pictures in the L0 reference picture list. That is, the prediction weighting table syntax may include the same number of luma_weight_l0_flag and / or chroma_weight_l0_flag as the number of weighted reference pictures in the L0 reference picture list, and may include the same number of luma_weight_l1_flag and / or chroma_weight_l1_flag as the number of weighted reference pictures in the L1 reference picture list.

[0252] The encoding device may encode the image information including the quantity information and the weighting factor related flag, and may output the encoded image information in the form of a bitstream. Here, the quantity information and the weighting factor related flag may be included in the prediction weighting table syntax in the image information as shown in Table 11. The prediction weighting table syntax may be included in the picture header in the image information or in the slice header of the image information. In order to indicate whether the prediction weighting table syntax is included in the picture header, that is, to indicate whether the information about weighted prediction is present in the picture header, the weighted prediction related flag may be included in the picture parameter set and / or the picture header. When the weighted prediction related flag is included in the picture parameter set, the weighted prediction related flag may correspond to pps_weighted_pred_flag of Table 5. When the weighted prediction related flag is included in the picture header, the weighted prediction related flag may correspond to pic_weighted_pred_flag of Table 7. Alternatively, pps_weighted_pred_flag and pic_weighted_pred_flag may be included in the image information to indicate whether the prediction weighting table syntax is included in the picture header.

[0253] When the weighted prediction related flag is parsed from the bitstream, the decoding device can parse the prediction weighting table syntax from the bitstream based on the parsed flag. The weighted prediction related flag can be parsed from the picture parameter set and / or the picture header of the bitstream. In other words, the weighted prediction related flag may include pps_weighted_pred_flag and / or pic_weighted_pred_flag. When the value of pps_weighted_pred_flag and / or pic_weighted_pred_flag is 1, the decoding device can parse the prediction weighting table syntax from the picture header of the bitstream.

[0254] When the prediction weighting table syntax is parsed from the picture header (when the value of pps_weighted_pred_flag and / or pic_weighted_pred_flag is 1), the decoding device may apply the information on weighted prediction included in the prediction weighting table syntax to all slices in the current picture. In other words, when the prediction weighting table syntax is parsed from the picture header, all slices in the picture associated with the picture header may have the same reference picture list.

[0255] The decoding device may parse the number information of the weighted reference pictures in the reference picture list of the current block based on the prediction weighting table syntax. The value of the number information may be equal to the number of weighted reference pictures in the reference picture list. When weighted bi-prediction is applied to the current block, the decoding device may independently parse the number information of the weighted reference pictures in the L1 reference picture list and the number information of the weighted reference pictures in the L0 reference picture list from the prediction weighting table syntax.

[0256] The decoding device may parse the weighting factor related flags of the reference picture list from the prediction weighting table syntax based on the quantity information. The weighting factor related flags may correspond to luma_weight_l0_flag, luma_weight_l1_flag, chroma_weight_l0_flag and / or chroma_weight_l0_flag of Table 11. For example, when the value of the quantity information is n, the decoding device may parse n weighting factor related flags from the prediction weighting table syntax. The decoding device may derive the weight of the reference picture of the current block based on the weighting factor related flags, and may perform weighted prediction on the current block based on the weights, thereby generating or deriving prediction samples. Subsequently, the decoding device may generate or derive reconstructed samples of the current block based on the prediction samples, and may reconstruct the current picture based on the reconstructed samples.

[0257] In another embodiment, for weighted prediction in inter-frame prediction, the picture header may include the syntax elements shown in Table 13 below, and the semantics of the syntax elements may be as shown in Table 14 below.

[0258] [Table 13]

[0259]

[0260] [Table 14]

[0261]

[0262] Referring to Table 13 and Table 14, a value of pic_weighted_pred_flag equal to 0 may indicate that weighted prediction is not applied to the P or B slice of the reference picture header. A value of pic_weighted_pred_flag equal to 1 may indicate that weighted prediction is applied to the P or B slice of the reference picture header. When the value of sps_weighted_pred_flag is 0, the value of pic_weighted_pred_flag is 0.

[0263] The slice header may include the syntax elements shown in Table 15 below.

[0264] [Table 15]

[0265]

[0266]

[0267] Referring to Table 15, the weighted prediction related flag (pic_weighted_pred_flag) may indicate whether the prediction weighting table syntax (information about weighted prediction) exists in the picture header or the slice header. A value of pic_weighted_pred_flag equal to 1 may indicate that the prediction weighting table syntax (information about weighted prediction) may exist in the picture header instead of the slice header.

[0268] A value of pic_weighted_pred_flag equal to 0 may indicate that the prediction weighting table syntax (information about weighted prediction) may be present in the slice header instead of the picture header. Although Tables 13 and 14 indicate that the weighted prediction related flags are signaled in the picture header, the weighted prediction related flags may also be signaled in the picture parameter set.

[0269] For example, when weighted prediction is applied to the current block, the encoding device performs weighted prediction, and can encode the image information including the weighted prediction related flag and the prediction weight table syntax based on the weighted prediction. Here, when the prediction weight table syntax is included in the picture header of the image information, the encoding device can determine that the value of the flag is 1, and when the prediction weight table syntax is included in the slice header of the image information, the encoding device can determine that the value of the flag is 0. When the value of the flag is 1, the information about weighted prediction included in the prediction weight table syntax can be applied to all slices in the current picture. When the value of the flag is 0, the information about weighted prediction included in the prediction weight table syntax can be applied to the slice associated with the slice header among the slices in the current picture. Therefore, when the prediction weight table syntax is included in the picture header, all slices in the picture associated with the picture header can have the same reference picture list, and when the prediction weight table syntax is included in the slice header, the slice associated with the slice header can have the same reference picture list.

[0270] The prediction weighting table syntax may include information about the number of weighted reference pictures in the reference picture list of the current block, weighting factor related flags, etc. As described above, the quantity information may refer to quantity information about the weights signaled for the items (reference pictures) in the L0 reference picture list and / or the L1 reference picture list, and the value of the quantity information may be equal to the number of weighted reference pictures in the reference picture list (L0 and / or L1). Therefore, when the value of the quantity information is n, the prediction weighting table syntax may include n weighting factor related flags for the reference picture list. The weighting factor related flags may correspond to luma_weight_l0_flag, luma_weight_l1_flag, chroma_weight_l0_flag and / or chroma_weight_l1_flag of Table 11.

[0271] chroma_weight_l0_flag.

[0272] When weighted bi-prediction is applied to the current block, the encoding device may generate a prediction weighting table syntax including information about the number of weighted reference pictures in the L1 reference picture list and information about the number of weighted reference pictures in the L0 reference picture list. The prediction weighting table syntax may independently include a weighting factor related flag for each of the information about the number of weighted reference pictures in the L1 reference picture list and the information about the number of weighted reference pictures in the L0 reference picture list. That is, the prediction weighting table syntax may include the same number of luma_weight_l0_flag and / or chroma_weight_l0_flag as the number of weighted reference pictures in the L0 reference picture list, and may include the same number of luma_weight_l1_flag and / or chroma_weight_l1_flag as the number of weighted reference pictures in the L1 reference picture list.

[0273] When a weighted prediction related flag is parsed from a bitstream, the decoding device may parse a prediction weighted table syntax from the bitstream based on the parsed flag. The weighted prediction related flag may be parsed from a picture parameter set and / or a picture header of the bitstream. In other words, the weighted prediction related flag may correspond to pps_weighted_pred_flag and / or pic_weighted_pred_flag. When the value of the weighted prediction related flag is 1, the decoding device may parse the prediction weighted table syntax from the picture header of the bitstream. When the value of the weighted prediction related flag is 0, the decoding device may parse the prediction weighted table syntax from the slice header of the bitstream.

[0274] When the prediction weighting table syntax is parsed from the picture header, the decoding device may apply the information about weighted prediction included in the prediction weighting table syntax to all slices in the current picture. In other words, when the prediction weighting table syntax is parsed from the picture header, all slices in the picture associated with the picture header may have the same reference picture list. When the prediction weighting table syntax is parsed from the slice header, the decoding device may apply the information about weighted prediction included in the prediction weighting table syntax to the slice associated with the slice header among the slices in the current picture. In other words, when the prediction weighting table syntax is parsed from the picture header, the slices associated with the slice header may have the same reference picture list.

[0275] The decoding device may parse the number information of the weighted reference pictures in the reference picture list of the current block based on the prediction weighting table syntax. The value of the number information may be equal to the number of weighted reference pictures in the reference picture list. When weighted bi-prediction is applied to the current block, the decoding device may independently parse the number information of the weighted reference pictures in the L1 reference picture list and the number information of the weighted reference pictures in the L0 reference picture list from the prediction weighting table syntax.

[0276] The decoding device may parse the weighting factor related flags of the reference picture list from the prediction weighting table syntax based on the quantity information. The weighting factor related flags may correspond to the above-mentioned luma_weight_l0_flag, luma_weight_l1_flag, chroma_weight_l0_flag and / or chroma_weight_l0_flag. For example, when the value of the quantity information is n, the decoding device may parse n weighting factor related flags from the prediction weighting table syntax. The decoding device may derive the weight of the reference picture of the current block based on the weighting factor related flags, and may perform inter-frame prediction on the current block based on the weights, thereby generating or deriving prediction samples. The decoding device may generate or derive reconstructed samples of the current block based on the prediction samples, and may generate a reconstructed picture of the current picture based on the reconstructed samples.

[0277] In yet another embodiment, the prediction weighting table syntax may include the syntax elements shown in Table 16 below, and the semantics of the syntax elements may be as shown in Table 17 below.

[0278] [Table 16]

[0279]

[0280] [Table 17]

[0281]

[0282]

[0283] In Tables 16 and 17, when pic_poc_delta_sign[i] does not exist, pic_poc_delta_sign[i] is inferred to be 0. DeltaPocWeightedRefPic[i] can be derived as follows, where i is included in the range from 0 to num_weighted_ref_pics_minus1.

[0284] [Equation 1]

[0285] DeltaPocWeightedRefPic[i] = pic_poc_abs_delta[i] * (1 - 2 * picpoc_delta_sign[i])

[0286] Chromaweight[i][j] can be derived as (1 << Chromalog2WeightDenom) + delta_chroma_weight[i][j]. When the value of chroma_weight_flag[i] is 1, the value of delta_chroma_weight[i][j] is included in the range from -128 to 127. When the value of chroma_weight_flag[i] is 0, Chromaweight[i][j] can be derived as 2Chromalog2WeightDenom.

[0287] ChromaOffset[i][j] can be derived as follows.

[0288] [Equation 2]

[0289] ChromaOffset[i][j] = Clip3(-128, 127, (128 + delta chroma offset[i][j] - ((128 * ChromaWeight[i][j]) >> ChromaLog2WeightDenom)))

[0290] The value of delta_chroma_offset[i][j] can be included in the range from -4 * 128 to 4 * 127. When the value of chroma_weight_flag[i] is 0, the value of ChromaOffset[i][j] is inferred to be 9.

[0291] sumWeightflags may be derived as the sum of luma_weight_flag[i]+2*chroma_weight_flag[i]. i is in the range from 0 to num_weighted_ref_pics_minus1, inclusive. When slice_type is P, sumWeightL0Flags is less than or equal to 24.

[0292] When the current slice is a P slice or a B slice and the value of pic_weighted_pred_flag is 1, L0ToWeightedRefIdx[i] may represent the mapping between the index in the weighted reference picture list and the i-th reference picture L0. i is included in the range from 0 to NumRefIdxActive[0]-1 and may be derived as follows.

[0293] [Formula 3]

[0294]

[0295] When the current slice is a B slice and the value of pic_weighted_pred_flag is 1, L1ToWeightedRefIdx[i] may represent the mapping between the index in the weighted reference picture list and the i-th active reference picture L1. i is included in the range from 0 to NumRefIdxActive[1]-1 and may be derived as follows.

[0296] [Formula 4]

[0297]

[0298] When luma_weight_l0_flag[i] appears, luma_weight_l0_flag[i] is replaced by luma_weight_flag[L0ToWeightedRefIdx[i]], and when luma_weight_l1_flag[i] appears, luma_weight_l1_flag[i] is replaced by luma_weight_flag[L1ToWeightedRefIdx[i]].

[0299] When LumaWeightL0[i] appears, LumaWeightL0[i] is replaced by LumaWeight[L0ToWeightedRefIdx[i]], and when LumaWeightL1[i] appears, LumaWeightL1[i] is replaced by LumaWeight[L1ToWeightedRefIdx[i]].

[0300] When luma_offset_l0[i] appears, luma_offset_l0[i] is replaced by luma_offset[L0ToWeightedRefIdx[i]], and when luma_offset_l1[i] appears, luma_offset_l1[i] is replaced by luma_offset[L1ToWeightedRefIdx[i]].

[0301] When ChromaWeightL0[i] occurs, ChromaWeightL0[i] is replaced by ChromaWeight[L0ToWeightedRefIdx[i]], and when ChromaWeightL1[i] occurs, ChromaWeightL1[i] is replaced by ChromaWeight[L1ToWeightedRefIdx[i]].

[0302] In yet another embodiment, the slice header syntax may include the syntax elements shown in Table 18 below, and the semantics of the syntax elements may be as shown in Table 19 below.

[0303] [Table 18]

[0304]

[0305] [Table 19]

[0306]

[0307] Referring to Table 18 and Table 19, a flag indicating whether a prediction weight table syntax is present in a slice header may be signaled. The flag may be signaled in a slice header and may be referred to as slice_weight_pred_present_flag.

[0308] A value of slice_weight_pred_present_flag equal to 1 may indicate that the prediction weighting table syntax is present in the slice header. A value of slice_weight_pred_present_flag equal to 0 may indicate that the prediction weighting table syntax is not present in the slice header. That is, slice_weight_pred_present_flag equal to 0 may indicate that the prediction weighting table syntax is present in the picture header.

[0309] In yet another embodiment, the prediction weighting table syntax is parsed from the slice header, but an adaptation parameter set including the syntax elements shown in Table 20 below may be signaled.

[0310] [Table 20]

[0311]

[0312] Each APS RBSP needs to be available to the decoding process before being included for use as a reference in at least one access unit whose TemporalId is less than or equal to the TemporalId of the coded slice NAL unit that references the APS RBSP or is provided by an external method.

[0313] aspLayerId may be referred to as the nuh_layer_id of the APS NAL unit. When the layer with nuh_layer_id equal to aspLayerId is an independent layer (i.e., when vps_independent_layer_flag[GeneralLayerIdx[aspLayerId]] is 1), the APS NAL unit including the APS RBSP has the same nuh_layer_id as the nuh_layer_id of the coded slice NAL referencing the APS RBSP. Otherwise, the APS NAL unit including the APS RBSP has the same nuh_layer_id as the nuh_layer_id of the coded slice NAL unit referencing the APS RBSP or the nuh_layer_id of a directly dependent layer of the layer including the coded slice NAL unit referencing the APS RBSP.

[0314] All APS NAL units in an access unit with a specific value of adaptation_parameter_set_id and a specific value of aps_params_type have the same content.

[0315] adaptation_parameter_set_id provides an identifier of the APS so that other syntax elements can refer to the identifier.

[0316] When aps_params_type is ALF_APS, SCALING_APS, or PRED_WEIGHT_APS, the value of adaptation_parameter_set_id is in the range from 0 to 7, included.

[0317] When aps_params_type is LMCS_APS, the value of adaptation_parameter_set_id is in the range from 0 to 3, included.

[0318] aps_params_type indicates the type of APS parameters included in the APS, as shown in the following Table 21. When the value of aps_params_type is 1 (LMCS_APS), the value of adaptation_parameter_set_id is included in the range from 0 to 3.

[0319] [Table 21]

[0320] aps_params_type The name of aps_params_type Types of APS parameters 0 ALF_APS ALF parameters 1 LMCS_APS LMCS parameters 2 SCALING_APS Scaling List Parameters 3 PRED_WEIGHT_APS Prediction weighting parameters 4..7 reserve reserve

[0321] Each type of APS uses a separate value space for adaptation_parameter_set_id.

[0322] APS NAL units (with specific values ​​of adaptation_parameter_set_id and specific values ​​of aps_params_type) can be shared between pictures, and different slices in a picture can reference different ALF APSs.

[0323] A value of aps_extension_flag equal to 0 indicates that the aps_extension_data_flag syntax element is not present in the APS RBSP syntax structure. A value of aps_extension_flag equal to 1 indicates that the aps_extension_data_flag syntax element is present in the APS RBSP syntax structure.

[0324] aps_extension_data_flag may have a random value.

[0325] As described above, a new aps_params_type (PRED_WEIGHT_APS) may be added to the existing types. In addition, the slice header may be modified to signal the APS ID instead of pred_weight_table(), as shown in Table 22 below.

[0326] [Table 22]

[0327]

[0328] In Table 22, slice_pred_weight_aps_id indicates adaptation_parameter_set_id of the prediction weight table APS. The TemporalId of the APS NAL unit having the same aps_params_type as PERD_WEIGHT_APS and the same adaptation_parameter_set_id as slice_pred_weight_aps_id is less than or equal to the TemporalId of the coded slice NAL unit.

[0329] When the slice_pred_weight_aps_id syntax element is present in the slice header, the value of slice_pred_weight_aps_id is the same for all slices of the picture.

[0330] In this case, the prediction weighting table syntax shown in Table 23 below may be signaled.

[0331] [Table 23]

[0332]

[0333] In Table 23, a value of num_lists_active_flag equal to 1 may indicate that prediction weight table information is signaled for one reference picture list. A value of num_lists_active_flag equal to 0 may indicate that prediction weight table information of two reference picture lists L0 and L1 is not signaled.

[0334] numRefIdxActive[i] may be used to indicate the number of active reference indices. The value of numRefIdxActive[i] ranges from 0 to 14.

[0335] The syntax of Table 23 indicates whether information about one or two lists is parsed in APS when num_lists_active_flag is parsed.

[0336] Instead of Table 23, the prediction weighting table syntax shown in Table 24 below may be used.

[0337] [Table 24]

[0338]

[0339] In Table 24, a value of num_lists_active_flag equal to 1 may indicate that prediction weighting table information is signaled for one reference picture list. A value of num_lists_active_flag equal to 0 may indicate that prediction weighting table information for two reference picture lists is not signaled.

[0340] In yet another embodiment, the picture parameter set syntax may include syntax elements as shown in Table 25 below, and the semantics of the syntax elements may be as shown in Table 26 below.

[0341] [Table 25]

[0342]

[0343] [Table 26]

[0344]

[0345] Referring to Table 25 and Table 26, the picture parameter set may include information about a flag indicating whether weighted prediction is applied to a slice referring to the picture parameter set and information about a flag indicating whether information about weighted prediction exists in a picture header or a slice header.

[0346] The information about the flag indicating whether weighted prediction is applied to a slice of a reference picture parameter set may include a pps_weighted_pred_flag syntax element indicating whether weighted prediction is applied to a P or B slice of the reference picture parameter set and / or a pps_weighted_bipred_flag syntax element indicating whether weighted prediction is applied to a B slice of the reference picture parameter set.

[0347] Information of a flag indicating whether information about weighted prediction is present in a picture header or a slice header may include a weighted_pred_table_present_in_ph_flag syntax element, and when a value of the pps_weighted_pred_flag and / or pps_weighted_bipred_flag syntax element is 1, the weighted_pred_table_present_in_ph_flag syntax element may be included in the picture parameter set. When a value of the flag indicating whether information about weighted prediction is present in a picture header or a slice header is 1, information about weighted prediction included in the prediction weighting table syntax may not be present in a slice header of the reference picture parameter set, but may be present in a picture header of the reference picture parameter set. When a value of the flag indicating whether information about weighted prediction is present in a picture header or a slice header is 0, information about weighted prediction may not be present in a picture header of the reference picture parameter set, but may be present in a slice header of the reference picture parameter set.

[0348] In this case, the picture header syntax and the slice header syntax may be signaled as shown in Table 27 and Table 28 below, respectively.

[0349] [Table 27]

[0350]

[0351] [Table 28]

[0352]

[0353] Referring to Table 27, when the value of the flag (pps_weighted_pred_flag and / or pps_weighted_bipred_flag) indicating whether weighted prediction is applied to the slice of the reference picture parameter set is 1 and the value of the flag (weighted_pred_table_present_in_ph_flag) indicating whether the information about weighted prediction is present in the picture header or the slice header is 1, the picture header syntax may include the prediction weighting table syntax (pred_weight_table).

[0354] Referring to Table 28, when the value of the flag (pps_weighted_pred_flag and / or pps_weighted_bipred_flag) indicating whether weighted prediction is applied to a slice of a reference picture parameter set is 1 and the value of the flag (weighted_pred_table_present_in_ph_flag) indicating whether information about weighted prediction is present in a picture header or a slice header is 0, the slice header syntax may include a prediction weighting table syntax (pred_weight_table). Specifically, when

[0355] The prediction weighting table syntax may be included in the slice header when the value of pps_weighted_pred_flag is 1, the slice type is a P slice, and the value of weighted_pred_table_present_in_ph_flag is 0. Alternatively, the prediction weighting table syntax may be included in the slice header when the value of pps_weighted_bipred_flag is 1, the slice type is a B slice, and the value of weighted_pred_table_present_in_ph_flag is 0.

[0356] The prediction weighting table syntax may include the syntax elements shown in Table 29 below, and the semantics of the syntax elements may be as shown in Table 30 below.

[0357] [Table 29]

[0358]

[0359] [Table 30]

[0360]

[0361] Referring to Table 29 and Table 30, the prediction weighting table syntax may include quantity information for weighted prediction. The quantity information may indicate the number of weighted reference pictures in the reference picture list. In other words, the value of the quantity information may be the same as the number of weighted reference pictures in the reference picture list. The reference picture list may include an L0 reference picture list and an L1 reference picture list. For example, the quantity information for weighted prediction may include first quantity information (num_10_weighted_ref_pics) of the L0 reference picture list and second quantity information (num_10_weighted_ref_pics) of the L1 reference picture list. As shown in Table 29, the first quantity information and the second quantity information may be parsed independently from the prediction weighting table syntax. The quantity information for weighted prediction may be included in the prediction weighting table syntax based on a flag (e.g., weighted_pred_table_present_in_ph_flag syntax element) indicating whether information about weighted prediction is present in a picture header or a slice header. For example, when the value of a flag indicating whether information about weighted prediction is present in a picture header or a slice header (e.g., a weighted_pred_table_present_in_ph_flag syntax element) is 1, quantity information for weighted prediction may be included in the prediction weight table syntax. When the value of flag is 0, the first quantity information may be inferred as NumRefIdxActive[0], and the second quantity information may be inferred as NumRefIdxActive[1]. NumRefIdxActive[i] may indicate the number of (active) reference indexes of RPL i used for decoding the corresponding slice.

[0362] A syntax element (luma_weight_10_flag) indicating whether a weighting factor for L0 prediction exists may be parsed from the prediction weighting table syntax based on the first quantity information. A syntax element (luma_weight_11_flag) indicating whether a weighting factor for L1 prediction exists may be parsed from the prediction weighting table syntax based on the second quantity information. For example, when the value of the first quantity information is n, n luma_weight_10_flag syntax elements may be parsed from the prediction weighting table syntax. Similarly, when the value of the second quantity information is n, n luma_weight_11_flag syntax elements may be parsed from the prediction weighting table syntax.

[0363] Fig. 9 and Fig.10 An example of a video / image encoding method and related components according to an embodiment of this document is schematically illustrated.

[0364] Fig. 9The video / image encoding method disclosed in Figure 2 and Fig.10 The (video / image) encoding device 200 disclosed in is executed. Specifically, for example, Fig. 9 S900 may be performed by the predictor 220 of the encoding apparatus 200, and S910 may be performed by the residual processor 230 of the encoding apparatus 200. S920 may be performed by the entropy encoder 240 of the encoding apparatus 200. Fig. 9 The video / image encoding method disclosed in may include the above-mentioned implementation methods of this document.

[0365] Specifically, refer to Fig. 9 and Fig.10 , the predictor 220 of the encoding device can derive motion information about the current block in the current picture based on motion estimation. For example, the encoding device can use the original block in the original picture to search for a similar reference block with high correlation relative to the current block in units of fractional pixels within a predetermined search range in the reference picture, and thus can derive motion information. The similarity of the blocks can be derived based on the difference between the sample values ​​based on the stage. For example, the similarity of the blocks can be calculated based on the sum of absolute differences (SAD) between the current block (or the template of the current block) and the reference block (or the template of the reference block). In this case, the motion information can be derived based on the reference block with the minimum SAD in the search area. According to various methods, based on the inter-frame prediction mode, the derived motion information can be notified to the decoding device with a signal.

[0366] The predictor 220 of the encoding device may perform weighted (sample) prediction on the current block based on the motion information about the current block, and may generate a prediction sample (prediction block) and prediction related information of the current block based on the weighted prediction (S900). The prediction related information may include prediction mode information (merge mode, skip mode, etc.), information about motion information, information about weighted prediction, etc. The information about motion information may include candidate selection information (e.g., merge index, mvp flag, or mvp index), which is information for deriving motion vectors. In addition, the information about motion information may include information about the above-mentioned MVD and / or reference picture index information. In addition, the information about motion information may include information indicating whether L0 prediction, L1 prediction, or dual prediction is applied. For example, when the slice type of the current slice is a P slice or a B slice, the predictor 220 may perform weighted prediction on the current block in the current slice. Weighted prediction can be used not only when dual prediction is applied to the current block, but also when single prediction is applied to the current block.

[0367] The residual processor 230 of the encoding device may generate residual samples and residual information based on the predicted samples generated by the predictor 220 and the original picture (original block and original sample) (S910). Here, the residual information is information about the residual samples and may include information about (quantized) transform coefficients for the residual samples.

[0368] The adder (or reconstructor) of the encoding device may generate a reconstructed sample (a reconstructed picture, a reconstructed block, or a reconstructed sample array) by adding the residual sample generated by the residual processor 230 and the prediction sample generated by the predictor 220 .

[0369] The entropy encoder 240 of the encoding device may encode the image information including the prediction related information and the information about the weighted prediction generated by the predictor 220, the residual information generated by the residual processor 230, etc. (S920). Here, the information about the weighted prediction may include a flag related to whether the weighted prediction is applied to a slice of the picture parameter set of the reference image information (hereinafter referred to as a "first flag"), a flag related to whether the information about the weighted prediction is present in a picture header of the image information or a slice header of the image information (hereinafter referred to as a "second flag"), a prediction weighting table syntax, etc. The first flag may include a pps_weighted_pred_flag syntax element indicating whether the weighted prediction is applied to a P or B slice of the reference picture parameter set and / or a pps_weighted_bipred_flag syntax element indicating whether the weighted prediction is applied to a B slice of the reference picture parameter set. The second flag may be, for example, weighted_pred_table_present_in_ph_flag, which may be included in the picture parameter set when the value of the pps_weighted_pred_flag and / or pps_weighted_bipred_flag syntax element is 1. When the value of the second flag is 1, information about weighted prediction included in the prediction weighting table syntax may not be present in the slice header of the reference picture parameter set, but may be present in the picture header of the reference picture parameter set. When the value of the second flag is 0, information about weighted prediction may not be present in the picture header of the reference picture parameter set, but may be present in the slice header of the reference picture parameter set.

[0370] Based on the values ​​of the first flag (pps_weighted_pred_flag and / or pps_weighted_bipred_flag) and the second flag, the prediction weighting table syntax may be included in a picture header or a slice header of the image information. For example, when the value of the first flag and the value of the second flag are both 1, the prediction weighting table syntax may be included in the picture header. When the value of the first flag is 1 and the value of the second flag is 0, the prediction weighting table syntax may be included in the slice header.

[0371] The prediction weighting table syntax may include quantity information for weighted prediction based on the value of the second flag. The value of the quantity information may be the same as the number of weighted reference pictures in the reference picture list. Here, the reference picture list may include an L0 reference picture list and an L1 reference picture list. The quantity information for weighted prediction may include first quantity information (num_10_weighted_ref_pics) about the L0 reference picture list and second quantity information (num_10_weighted_ref_pics) about the L1 reference picture list. As shown in Table 29, the first quantity information and the second quantity information may be parsed independently from the prediction weighting table syntax. Based on whether the value of the second flag is 0 or 1, the quantity information for weighted prediction may be included in the prediction weighting table syntax or may not be included in the prediction weighting table syntax. For example, when the value of the second flag is 1, the quantity information for weighted prediction may be included in the prediction weighting table syntax. When the value of the second flag is 0, the first quantity information may be inferred to be NumRefIdxActive[0], and the second quantity information may be inferred to be NumRefIdxActive[1].

[0372] The prediction weight table syntax may include a syntax element indicating whether a weighting factor for L0 prediction exists

[0373] (luma_weight_10_flag) and / or a syntax element indicating whether a weighting factor for L1 prediction exists (luma_weight_11_flag). luma_weight_10_flag can be included in the prediction weighting table syntax based on the first quantity information. luma_weight_11_flag can be included in the prediction weighting table syntax based on the second quantity information. For example, when the value of the first quantity information is n, n luma_weight_10_flag syntax elements can be included in the prediction weighting table syntax. Similarly, when the value of the second quantity information is n, n luma_weight_11_flag syntax elements can be included in the prediction weighting table syntax.

[0374] For example, the entropy encoder 240 of the encoding device may encode the image information based on at least one of Tables 5 to 23, and may output the encoded image information in the form of a bitstream. Specifically, when the prediction weighting table syntax is included in the picture header of the image information, the entropy encoder 240 of the encoding device may determine the value of the first flag (pps_weighted_pred_flag and / or pps_weighted_bipred_flag) and the value of the second flag (weighted_pred_table_present_in_ph_flag) to be 1. When the prediction weighting table syntax is included in the slice header of the image information, the entropy encoder 240 of the encoding device may determine the value of the first flag to be 1 and may determine the value of the second flag to be 0. When the prediction weighting table syntax is included in the picture header, all slices associated with the picture header in the picture may have the same reference picture list, and when the prediction weighting table syntax is included in the slice header, the slice associated with the slice header may have the same reference picture list. The first flag and the second flag may be included in the picture parameter set of the image information and transmitted to the decoding device.

[0375] The predictor 220 of the encoding device may generate quantity information about weighted reference pictures in the reference picture list based on weighted prediction according to motion information. In this case, the entropy encoder 240 of the encoding device may encode the image information including information about the quantity. The quantity information may be included in the prediction weighting table syntax in the image information or not included in the prediction weighting table syntax in the image information based on whether the value of the flag is 0 or 1. The value of the quantity information may be the same as the number of weighted reference pictures in the reference picture list. Therefore, the prediction weighting table syntax may include as many weighting factor related flags (luma_weight_l0_flag and / or luma_weight_l1_flag) as the value of the quantity information. For example, when the value of the quantity information is n, the prediction weighting table syntax may include n weighting factor related flags. For each of L0 and L1, the quantity information and / or weighting factor related flags may be independently included in the prediction weighting table syntax. In other words, information about the number of weighted reference pictures in L0 and information about the number of weighted reference pictures in L1 can be signaled independently in the prediction weighting table syntax without depending on each other (not depending on the number of active reference pictures of each list).

[0376] Fig.11 and Fig.12 An example of a video / image decoding method and related components according to an embodiment of this document is schematically illustrated.

[0377] Fig.11The video / image decoding method disclosed in Figure 3 and Fig.12 The (video / image) decoding device 300 disclosed in is executed. Specifically, for example, Fig.11 S1100 and S1110 of the decoding device may be performed by the entropy decoder 310 of the decoding device. S1120 may be performed by the predictor 330 of the decoding device, and S1130 may be performed by the adder 340 of the decoding device. Fig.11 The video / image decoding method disclosed in may include the above-mentioned implementation modes of this document.

[0378] refer to Fig.11 and Fig.12 , the entropy decoder 310 of the decoding device may parse a flag related to weighted prediction from a picture parameter set of a bitstream (S1100). The flag may include a first flag related to whether weighted prediction is applied to a slice of a reference picture parameter set and a second flag related to whether information about weighted prediction is present in a picture header of a bitstream or a slice header of a bitstream. The entropy decoder 310 of the decoding device may parse the prediction weighting table syntax from a picture header or a slice header based on the flags (first flag and second flag) (S1110). Here, the first flag may include a pps_weighted_pred_flag syntax element indicating whether weighted prediction is applied to a P or B slice of a reference picture parameter set and / or a pps_weighted_bipred_flag syntax element indicating whether weighted prediction is applied to a B slice of a reference picture parameter set. The second flag may be parsed based on the first flag. For example, when the value of the pps_weighted_pred_flag and / or pps_weighted_bipred_flag syntax element is 1, the second flag may be parsed from the picture parameter set.

[0379] When the value of the second flag is 1, the information about weighted prediction included in the prediction weighting table syntax may not exist in the slice header of the reference picture parameter set, but may exist in the picture header of the reference picture parameter set. When the value of the second flag is 0, the information about weighted prediction may not exist in the picture header of the reference picture parameter set, but may exist in the slice header of the reference picture parameter set. Therefore, when the value of the first flag and the value of the second flag are 1, the entropy decoder 310 of the decoding device can parse the prediction weighting table syntax from the picture header of the bitstream. When the value of the first flag is 1 and the value of the second flag is 0, the entropy decoder 310 of the decoding device can parse the prediction weighting table syntax from the slice header of the bitstream. When the prediction weighting table syntax is parsed from the picture header, all slices associated with the picture header in the picture can have the same reference picture list, and when the prediction weighting table syntax is parsed from the slice header, the slices associated with the slice header can have the same reference picture list.

[0380] The entropy decoder 310 of the decoding device can parse the quantity information for weighted prediction from the prediction weighting table syntax. The value of the quantity information can be the same as the number of weighted reference pictures in the reference picture list. The entropy decoder 310 of the decoding device can parse as many weighting factor related flags (luma_weight_l0_flag and / or luma_weight_l1_flag) as the value of the quantity information from the prediction weighting table syntax based on the quantity information. For example, when the value of the second flag is 1, the entropy decoder 310 of the decoding device can parse the quantity information from the prediction weighting table syntax. When the value of the second flag is 0, the first quantity information can be inferred to be NumRefIdxActive[0], and the second quantity information can be inferred to be NumRefIdxActive[1]. When the value of the quantity information is n, n weighting factor related flags can be parsed from the prediction weighting table syntax. For each of L0 and L1, the quantity information and / or weighting factor related flags can be independently included in the prediction weighting table syntax. In one example, information about the number of weighted reference pictures in L0 and information about the number of weighted reference pictures in L1 can be parsed independently in the prediction weighting table syntax without depending on each other (not depending on the number of active reference pictures of each list).

[0381] The decoding device may perform weighted prediction on the current block in the current picture based on the prediction related information (inter / intra prediction classification information, intra prediction mode information, inter prediction mode information, information about weighted prediction, etc.) obtained from the bitstream, thereby reconstructing the current picture. Specifically, the predictor 330 of the decoding device may perform weighted prediction on the current block based on the syntax elements in the prediction weighting table syntax, thereby generating a prediction sample of the current block (S1120). In one example, the predictor 330 of the decoding device may parse a weighting factor related flag based on the quantity information for weighted prediction in the prediction weighting table syntax, and may derive the weight of the weighted prediction based on the weighting factor related flag. For example, when the value of the quantity information is n, the predictor 330 of the decoding device may parse n weighting factor related flags from the prediction weighting table syntax. The predictor 330 of the decoding device may perform weighted prediction on the current block based on the weight, thereby deriving the prediction sample of the current block.

[0382] The residual processor 320 of the decoding device may generate residual samples based on the residual information obtained from the bitstream. The adder 340 of the decoding device may generate reconstructed samples based on the predicted samples generated by the predictor 330 and the residual samples generated by the residual processor 320 (S1130). The adder 340 of the decoding device may generate a reconstructed picture (reconstructed block) based on the reconstructed samples.

[0383] Subsequently, if necessary, an in-loop filtering process (such as deblocking filtering, SAO and / or ALF) may be applied to the reconstructed picture to improve the subjective / objective picture quality.

[0384] Although the method has been described based on a flowchart in which steps or boxes are listed in sequence in the above-mentioned embodiments, the steps of this document are not limited to a specific order, and a certain step may be performed in a different step or in a different order or simultaneously relative to the above-mentioned order. In addition, it should be understood by a person of ordinary skill in the art that the steps in the flowchart are not exclusive, and another step may be included or one or more steps in the flowchart may be deleted without affecting the scope of this document.

[0385] The above-mentioned method according to the present document may be in the form of software, and the encoding device and / or decoding device according to the present document may be included in a device for performing image processing (e.g., a TV, a computer, a smart phone, a set-top box, a display device, etc.).

[0386] When the implementation of this document is implemented by software, the above-mentioned method can be implemented by a module (processing or function) that performs the above-mentioned functions. The module can be stored in a memory and executed by a processor. The memory can be installed inside or outside the processor and can be connected to the processor via various well-known devices. The processor may include an application-specific integrated circuit (ASIC), other chipsets, logic circuits and / or data processing devices. The memory may include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium and / or other storage devices. In other words, according to the implementation of this document, it can be implemented and executed on a processor, a microprocessor, a controller or a chip. For example, the functional units illustrated in the corresponding figure can be implemented and executed on a computer, a processor, a microprocessor, a controller or a chip. In this case, information about the implementation (e.g., information about instructions) or an algorithm can be stored in a digital storage medium.

[0387] In addition, the decoding device and the encoding device applying the embodiments of this document can be included in a multimedia broadcast transceiver, a mobile communication terminal, a home theater video device, a digital theater video device, a surveillance camera, a video chat device, a real-time communication device such as video communication, a mobile streaming device, a storage medium, a portable camera, a video on demand (VoD) service provider, an over-the-top (OTT) video device, an Internet streaming service provider, a 3D video device, a virtual reality (VR) device, an augmented reality (AR) device, an image phone video device, a vehicle terminal (e.g., a vehicle (including an autonomous vehicle) terminal, an aircraft terminal or a ship terminal) and a medical video device; and can be used to process image signals or data. For example, an OTT video device may include a game console, a Blueray player, a networked TV, a home theater system, a smart phone, a tablet PC, and a digital video recorder (DVR).

[0388] In addition, the processing method of the embodiment of the present document can be generated in the form of a program executed by a computer and can be stored in a computer-readable recording medium. The multimedia data with a data structure according to the embodiment of the present document can also be stored in a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices storing computer-readable data. The computer-readable recording medium may include, for example, a Blu-ray disc (BD), a universal serial bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The computer-readable recording medium also includes a medium implemented in the form of a carrier wave (e.g., transmission on the Internet). In addition, the bit stream generated by the encoding method can be stored in a computer-readable recording medium, or can be transmitted through a wired or wireless communication network.

[0389] In addition, the embodiments of the present document can be implemented as a computer program product based on a program code, and the program code can be executed on a computer according to the embodiments of the present document. The program code can be stored on a computer readable carrier.

[0390] Fig.13 represents an example of a content streaming system to which embodiments of this document may be applied.

[0391] refer to Fig.13 A content streaming system to which embodiments of this document are applied may generally include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0392] The encoding server is used to compress the content input from the multimedia input device such as a smart phone, a camera, a camcorder, etc. into digital data, generate a bit stream, and transmit it to the streaming server. In another example, in the case where the multimedia input device such as a smart phone, a camera, a camcorder, etc. directly generates a bit stream, the encoding server can be omitted.

[0393] The bitstream may be generated by an encoding method or a bitstream generation method to which the embodiments of this document are applied, and the streaming server may temporarily store the bitstream in the process of sending or receiving the bitstream.

[0394] The streaming server transmits multimedia data to the user device based on the user's request through the network server, which acts as a tool to inform the user of what services exist. When the user requests the service the user wants, the network server transfers the request to the streaming server, and the streaming server transmits the multimedia data to the user. In this regard, the content streaming system may include a separate control server, and in this case, the control server is used to control the commands / responses between the various devices in the content streaming system.

[0395] The streaming server may receive content from a media storage and / or encoding server. For example, in the case where content is received from an encoding server, the content may be received in real time. In this case, the streaming server may store the bitstream for a predetermined period of time to smoothly provide a streaming service.

[0396] For example, user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation, tablet PCs, tablet PCs, ultrabooks, wearable devices (e.g., watch-type terminals (smart watches), glasses-type terminals (smart glasses), head-mounted displays (HMDs)), digital TVs, desktop computers, digital signage, etc.

[0397] Each server in the content streaming system may be operated as a distributed server, and in this case, data received by each server may be processed in a distributed manner.

Claims

1. A decoding device for image decoding, the decoding device comprising: Memory; as well as at least one processor, the at least one processor connected to the memory, the at least one processor configured to: Parse weighted prediction related flags from the bitstream; Parsing a prediction weighting table syntax from a picture header of the bitstream based on the flag; Generate a prediction sample for a current block in a current picture based on a syntax element in the prediction weighting table syntax; as well as Generate a reconstructed sample based on the predicted sample, The flag includes a first flag related to whether the weighted prediction is applied to a slice that refers to a picture parameter set of the bitstream and a second flag related to whether information about the weighted prediction is present in the picture header, wherein the information about the weighted prediction includes the prediction weighting table syntax, The second flag is parsed from the picture parameter set based on the first flag, The current picture includes multiple slices, and The prediction weighting table syntax included in the picture header is related to all the multiple slices in the current picture that references the picture header.

2. A coding device for image coding, the coding device comprising: Memory; as well as at least one processor, the at least one processor connected to the memory, the at least one processor configured to: generating a prediction sample for a current block in a current picture by performing weighted prediction based on motion information about the current block; Generate residual information based on the predicted sample and the original sample of the current picture; as well as encoding image information including a flag related to the weighted prediction, information about the weighted prediction, and the residual information, wherein the information about the weighted prediction includes a prediction weighting table syntax, wherein the flag includes a first flag related to whether the weighted prediction is applied to a slice of a picture parameter set that references the image information and a second flag related to whether the information about the weighted prediction exists in a picture header of the image information, wherein the second flag is included in the picture parameter set based on the first flag, wherein the prediction weighting table syntax is included in the picture header based on the flag, The current picture includes multiple slices, and The prediction weighting table syntax included in the picture header is related to all the multiple slices in the current picture that references the picture header.

3. A device for transmitting data for an image, the device comprising: at least one processor configured to obtain a bitstream for the image, wherein the bitstream is generated based on the following operations: generating a prediction sample for the current block by performing weighted prediction based on motion information of the current block in the current picture, generating residual information based on the prediction sample and an original sample of the current picture, and encoding image information including a flag related to the weighted prediction, information about the weighted prediction, and the residual information; and a transmitter configured to transmit the data including the bit stream, wherein the information about the weighted prediction includes a prediction weighting table syntax, The flag includes a first flag related to whether the weighted prediction is applied to a slice of a picture parameter set that references the image information and a second flag related to whether information about the weighted prediction exists in a picture header of the image information, wherein the second flag is included in the picture parameter set based on the first flag, wherein the prediction weighting table syntax is included in the picture header based on the flag, The current picture includes multiple slices, and The prediction weighting table syntax included in the picture header is related to all the multiple slices in the current picture that references the picture header.

Images