Syntax design method and coding device using syntax

Abstract

To provide an image decoding method performed by a decoding device.SOLUTION: The method comprises the steps of: decoding, on the basis of a bit stream, an affine flag that indicates whether or not an affine prediction can be applied to a current block and a sub-block TMVP flag that indicates whether a temporal motion vector predictor based on a sub-block of the current block is usable; determining whether or not to decode a predetermined marge mode flag that indicates whether or not to apply a predetermined marge mode to the current block; deriving prediction samples for the current block; and generating reconstructed samples for the current block on the basis of the prediction samples for the current block. In the case where a value of the affine flag is 1 or a value of the sub-block TMVP flag is 1, it is determined to decode the predetermined merge mode flag.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to image coding technology, and more particularly to a syntax design method and an apparatus for coding using the syntax in an image coding system. [Background technology]

[0002] In recent years, the demand for high-resolution, high-quality images / videos such as 4K or 8K or higher UHD (Ultra High Definition) images / videos is increasing in various fields. As the image / video data has higher resolution and quality, the amount of information or bits to be transmitted increases relatively compared to existing image / video data, so that when image data is transmitted using a medium such as an existing wired or wireless broadband line or image / video data is stored using an existing storage medium, the transmission cost and storage cost increase.

[0003] In addition, in recent years, interest in and demand for immersive media such as virtual reality (VR), artificial reality (AR) content, and holograms has been increasing, and the broadcast of images / videos having different image characteristics from real images, such as game images, has been increasing.

[0004] Therefore, there is a demand for highly efficient image / video compression technology to effectively compress and transmit, store, and play back high-resolution, high-quality image / video information having the various characteristics described above. Summary of the Invention [Problem to be solved by the invention]

[0005] A technical problem of the present disclosure is to provide a method and apparatus for improving image coding efficiency.

[0006] Another technical object of the present disclosure is to provide a syntax design method and an apparatus for coding using the syntax.

[0007] Yet another technical object of the present disclosure is to provide a high-level syntax and low-level syntax design method and an apparatus for coding using the syntax.

[0008] Yet another technical object of the present disclosure is to provide a method and apparatus using high-level and / or low-level syntax elements for performing motion prediction based on sub-blocks.

[0009] Yet another technical object of the present disclosure is to provide a method and apparatus using high-level and / or low-level syntax elements for performing motion prediction based on an affine model.

[0010] Yet another technical object of the present disclosure is to provide a method and apparatus for determining whether to decode a predetermined merge mode flag indicating whether to apply a predetermined merge mode to a current block based on an affine flag and a sub-block TMVP flag. [Means for solving the problem]

[0011] According to an embodiment of the present disclosure, an image decoding method performed by a decoding device is provided, the method includes the steps of: decoding an affine flag indicating whether affine prediction can be applied to a current block and a sub-block TMVP flag indicating whether a temporal motion vector predictor based on a sub-block of the current block can be used based on a bitstream; determining whether to decode a pre-determined merge mode flag indicating whether a pre-determined merge mode is applied to the current block based on the decoded affine flag and the decoded sub-block TMVP flag; deriving a prediction sample for the current block based on the decision on whether to decode the pre-determined merge mode flag; and generating a reconstructed sample for the current block based on the prediction sample for the current block, and is characterized in that if the value of the affine flag is 1 or the value of the sub-block TMVP flag is 1, it is determined to decode the pre-determined merge mode flag.

[0012] According to another embodiment of the present disclosure, there is provided a decoding device for performing image decoding, the decoding device including: an entropy decoding unit that decodes an affine flag indicating whether affine prediction can be applied to a current block and a sub-block TMVP flag indicating whether a temporal motion vector predictor based on a sub-block of the current block can be used based on a bitstream, and determines whether to decode a determined merge mode flag indicating whether a predetermined merge mode is applied to the current block based on the decoded affine flag and the decoded sub-block TMVP flag, a prediction unit that derives a prediction sample for the current block based on the determination of whether to decode the determined merge mode flag, and an adder that generates a reconstructed sample for the current block based on the prediction sample for the current block, and is characterized in that the decoding device determines to decode the determined merge mode flag when the value of the affine flag is 1 or the value of the sub-block TMVP flag is 1.

[0013] According to yet another embodiment of the present disclosure, there is provided an image encoding method performed by an encoding device, the method including the steps of: determining whether affine prediction is applicable to a current block and whether a temporal motion vector predictor based on a sub-block of the current block can be used; determining whether to encode a determined merge mode flag indicating whether a predetermined merge mode is applied to the current block based on the determination of whether the affine prediction is applicable to the current block and whether the temporal motion vector predictor based on the sub-block of the current block can be used; and encoding an affine flag indicating whether the affine prediction is applicable to the current block, a sub-block TMVP flag indicating whether the temporal motion vector predictor based on the sub-block of the current block can be used, and the determined merge mode flag based on the determination of whether to encode the determined merge mode flag, wherein if a value of the affine flag is 1 or a value of the sub-block TMVP flag is 1, it is determined to encode the determined merge mode flag.

[0014] According to yet another embodiment of the present disclosure, there is provided an encoding device for performing image encoding, comprising: a prediction unit for determining whether affine prediction is applicable to a current block and whether a temporal motion vector predictor based on a sub-block of the current block can be used, and determining whether to encode a determined merge mode flag indicating whether a predetermined merge mode is applied to the current block based on the determination of whether the affine prediction is applicable to the current block and whether the temporal motion vector predictor based on the sub-block of the current block can be used, and an entropy encoding unit for encoding an affine flag indicating whether the affine prediction is applicable to the current block, a sub-block TMVP flag indicating whether the temporal motion vector predictor based on the sub-block of the current block can be used, and the determined merge mode flag based on the determination of whether to encode the determined merge mode flag, wherein the encoding device is characterized in that if a value of the affine flag is 1 or a value of the sub-block TMVP flag is 1, it is determined to encode the determined merge mode flag.

[0015] According to yet another embodiment of the present disclosure, there is provided a decoder-readable storage medium storing information regarding instructions for causing a video decoding device to perform a decoding method according to some embodiments.

[0016] According to yet another embodiment of the present disclosure, a decoder-readable storage medium storing information on instructions for causing a video decoding device to perform a decoding method according to an embodiment is provided. The decoding method according to the embodiment includes the steps of: decoding an affine flag indicating whether affine prediction can be applied to a current block and a sub-block TMVP flag indicating whether a temporal motion vector predictor based on a sub-block of the current block can be used, based on a bitstream; determining whether to decode a determined merge mode flag indicating whether a predetermined merge mode is applied to the current block, based on the decoded affine flag and the decoded sub-block TMVP flag; deriving a prediction sample for the current block based on the decision on whether to decode the determined merge mode flag; and generating a reconstructed sample for the current block based on the prediction sample for the current block, and is characterized in that if the value of the affine flag is 1 or the value of the sub-block TMVP flag is 1, it is determined to decode the determined merge mode flag. Effect of the Invention

[0017] According to the present disclosure, the overall image / video compression efficiency can be improved.

[0018] According to the present disclosure, image coding efficiency can be increased through high-level and low-level syntax design.

[0019] According to the present disclosure, image coding efficiency can be increased by using high-level and / or low-level syntax elements for performing motion estimation on a sub-block basis.

[0020] According to the present disclosure, image coding efficiency can be increased by using high-level and / or low-level syntax elements for motion prediction based on an affine model.

[0021] According to the present disclosure, image coding efficiency can be improved by determining whether to decode a pre-determined merge mode flag, which indicates whether to apply a pre-determined merge mode to a current block, based on an affine flag and a sub-block TMVP flag. [Brief description of the drawings]

[0022] [Figure 1] 1 illustrates generally an example of a video / image coding system to which the present disclosure may be applied; [Diagram 2] A diagram that illustrates the configuration of a video / image encoding device to which the present disclosure can be applied. [Diagram 3] 1 is a diagram illustrating the schematic configuration of a video / image decoding device to which the present disclosure can be applied. [Figure 4] 1 is a flowchart illustrating an operation of an encoding device according to an embodiment. [Diagram 5] FIG. 1 is a block diagram showing a configuration of an encoding device according to an embodiment. [Figure 6] 10 is a flowchart illustrating an operation of a decoding device according to an embodiment. [Figure 7] 1 is a block diagram showing a configuration of a decoding device according to an embodiment; [Figure 8] 1 illustrates an example of a content streaming system to which the disclosure of this document may be applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] According to an embodiment of the present disclosure, an image decoding method performed by a decoding device is provided, the method includes the steps of: decoding an affine flag indicating whether affine prediction can be applied to a current block and a sub-block TMVP flag indicating whether a temporal motion vector predictor based on a sub-block of the current block can be used based on a bitstream; determining whether to decode a pre-determined merge mode flag indicating whether a pre-determined merge mode is applied to the current block based on the decoded affine flag and the decoded sub-block TMVP flag; deriving a prediction sample for the current block based on the decision on whether to decode the pre-determined merge mode flag; and generating a reconstructed sample for the current block based on the prediction sample for the current block, and is characterized in that if the value of the affine flag is 1 or the value of the sub-block TMVP flag is 1, it is determined to decode the pre-determined merge mode flag.

[0024] The present disclosure may be modified in various ways and may have various embodiments, and a specific embodiment will be illustrated in the drawings and described in detail. However, this is not intended to limit the present disclosure to a specific embodiment. Terms commonly used in this specification are used merely to describe a specific embodiment, and are not intended to limit the technical idea of ​​the present disclosure. A singular expression includes a plural expression unless the context clearly indicates otherwise. In this specification, the terms "include" or "have" are intended to specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0025] Meanwhile, each configuration in the drawings described in this disclosure is illustrated independently for the convenience of explaining the different characteristic functions, etc., and does not mean that each configuration is realized by separate hardware or software. For example, two or more of each configuration may be combined to form one configuration, and one configuration may be divided into multiple configurations. An embodiment in which each configuration is integrated and / or separated is also included in the scope of the present disclosure as long as it does not deviate from the essence of the present disclosure.

[0026] Hereinafter, preferred embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Hereinafter, the same reference numerals are used for the same components in the drawings, and duplicated descriptions of the same components may be omitted.

[0027] FIG. 1 illustrates generally an example of a video / image coding system to which the present disclosure may be applied.

[0028] As shown in Figure 1, a video / image coding system may include a first device (source device) and a second device (receiving device). The source device may transmit encoded video / image information or data to the receiving device in a file or streaming form via a digital storage medium or a network.

[0029] The source device may comprise a video source, an encoding device, and a sending unit. The receiving device may comprise a receiving unit, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be comprised in the encoding device. The receiver may be comprised in the decoding device. The renderer may comprise a display unit, which may be a separate device or an external component.

[0030] A video source may obtain video / images through a video / image capture, synthesis, or generation process. A video source may include a video / image capture device and / or a video / image generation device. A video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / images, etc. A video / image generation device may include, for example, a computer, a tablet, a smartphone, etc., and may (electronically) generate video / images. For example, a virtual video / image may be generated via a computer, etc., in which case the video / image capture process may be replaced by a process in which related data is generated.

[0031] An encoding device can encode an input video / image. The encoding device can perform a series of steps such as prediction, transformation, quantization, etc. for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0032] The transmitting unit may transmit the encoded video / image information or data output in the form of a bitstream to a receiving unit of a receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitting unit may include elements for generating a media file through a predetermined file format and may include elements for transmission via a broadcasting / communication network. The receiving unit may receive / extract the bitstream and transmit it to a decoding device.

[0033] The decoding device can decode the video / image by performing a series of steps such as inverse quantization, inverse transformation, prediction, etc., which correspond to the operations of the encoding device.

[0034] The renderer can render the decoded video / images. The rendered video / images can be displayed via a display unit.

[0035] This document relates to video / image coding. For example, the methods / embodiments disclosed in this document can be applied to methods disclosed in the versatile video coding (VVC) standard, the essential video coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the 2nd generation of audio video coding standard (AVS2), or the next generation video / image coding standard (e.g., H.267 or H.268, etc.).

[0036] In this document, various embodiments relating to video / image coding are presented, which may be performed in combination with each other, unless otherwise stated.

[0037] In this document, a video may mean a collection of a series of images over time. A picture generally means a unit showing one image at a specific time, and a slice / tile is a unit constituting a part of a picture in coding. A slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / tiles. A picture may be composed of one or more tile groups. A tile group may include one or more tiles. A brick may represent a rectangular region of CTU rows within a tile in a picture. A tile may be partitioned into multiple bricks, each of which consisting of one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks may be also referred to as a brick.A brick scan may refer to a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a brick, bricks within a tile are ordered consecutively in a raster scan of the bricks of the tile, and tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. A tile is a rectangular region of CTUs within a particular tile column and a particular tile row in a picture. The tile column is a rectangular region of CTUs having a height equal to the height of the picture and a width specified by syntax elements in the picture parameter set.The tile row is a rectangular region of CTUs having a height specified by syntax elements in the picture parameter set and a width equal to the width of the picture. A tile scan is a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a tile whereas tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. A slice includes an integer number of bricks of a picture that may be exclusively contained in a single NAL unit.A slice may consist of either a number of complete tiles or only a consecutive sequence of complete bricks of one tile. In this document, tile group and slice may be used interchangeably. For example, tile group / tile group header may be referred to as slice / slice header in this document.

[0038] A pixel or a pel may refer to the smallest unit that constitutes one picture (or image). A term corresponding to a pixel may be "sample." A sample may generally indicate a pixel or a pixel value, may indicate only a pixel / pixel value of a luma component, or may indicate only a pixel / pixel value of a chroma component.

[0039] A unit may indicate a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to the region. One unit may include one luma block and two chroma (e.g., cb, cr) blocks. A unit may be used in combination with terms such as block or area, depending on the case. In a general case, an M×N block may include a set (or array) of samples or transform coefficients consisting of M columns and N rows.

[0040] In this document, the term “ / ” and “,” should be interpreted to indicate “and / or.” For instance, the expression “A / B” may mean “A and / or B.” Further, “A,B” may mean “A and / or B.” Further, “A / B / C” may mean “at least one of A,B, and / or C.” Also, “A / B / C” may mean “at least one of A,B, and / or C.”

[0041] Further, in the document, the term “or” should be interpreted to indicate “and / or.” For instance, the expression “A or B” may comprise 1) only A, 2) only B, and / or 3) both A and B. In other words, the term “or” in this document should be interpreted to indicate “additionally or alternatively.”

[0042] 2 is a diagram for explaining the schematic configuration of a video / image encoding device to which the present disclosure can be applied. Hereinafter, the video encoding device may include an image encoding device.

[0043] As shown in FIG. 2, the encoding device 200 may be configured to include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter prediction unit 221 and an intra prediction unit 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. The image division unit 210, the prediction unit 220, the residual processing unit 230, the entropy encoding unit 240, the addition unit 250, and the filtering unit 260 may be configured by one or more hardware components (e.g., an encoder chip set or a processor) according to an embodiment. In addition, the memory 270 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware components may further include the memory 270 as an internal / external component.

[0044] The image division unit 210 may divide an input image (or picture, frame) input to the encoding device 200 into one or more processing units. As an example, the processing units may be called coding units (CUs). In this case, the coding units may be recursively divided from a coding tree unit (CTU) or a largest coding unit (LCU) according to a quad-tree binary-tree ternary-tree (QTBTTT) structure. For example, one coding unit may be divided into a plurality of coding units of a deeper depth based on a quad-tree structure, a binary tree structure, and / or a ternary structure. In this case, for example, the quad-tree structure may be applied first, and then the binary tree structure and / or the ternary structure may be applied. Alternatively, the binary tree structure may be applied first. The coding procedure according to the present disclosure may be performed based on a final coding unit that is not further divided. In this case, the maximum coding unit may be used as the final coding unit based on coding efficiency according to image characteristics, or the coding unit may be recursively divided into coding units of lower depths as necessary, and a coding unit of an optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later. As another example, the processing unit may further include a prediction unit (PU: Prediction Unit) or a transform unit (TU: Transform Unit). In this case, the prediction unit and the transform unit may each be divided or partitioned from the final coding unit described above.The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving a residual signal from the transform coefficients.

[0045] The unit may be used interchangeably with terms such as block or area. In general, an M×N block may refer to a collection of samples or transform coefficients consisting of M columns and N rows. A sample may generally refer to a pixel or pixel value, may refer to only a pixel / pixel value of a luma component, or may refer to only a pixel / pixel value of a chroma component. A sample may be used as a term that corresponds to a pixel or pel of one picture (or image).

[0046] The encoding device 200 may generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (predicted block, prediction sample array) output from the inter prediction unit 221 or the intra prediction unit 222 from an input image signal (original block, original sample array), and the generated residual signal is transmitted to the conversion unit 232. In this case, as illustrated, a unit in the encoder 200 that subtracts a prediction signal (predicted block, prediction sample array) from an input image signal (original block, original sample array) may be called a subtraction unit 231. The prediction unit may perform prediction on a block to be processed (hereinafter, referred to as a current block) and generate a predicted block including a prediction sample for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied in units of the current block or CU. The prediction unit may generate various information related to prediction, such as prediction mode information, and transmit the information to the entropy encoding unit 240, as will be described later in the description of each prediction mode. The prediction information can be encoded by the entropy encoding unit 240 and output in the form of a bitstream.

[0047] The intra prediction unit 222 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located away from the current block depending on the prediction mode. In the intra prediction, the prediction mode may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes depending on the fineness of the prediction direction. However, this is merely an example, and more or less directional prediction modes may be used depending on the settings. The intra prediction unit 222 may also determine the prediction mode to be applied to the current block using the prediction modes applied to the neighboring blocks.

[0048] The inter prediction unit 221 may derive a predicted block for a current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between a neighboring block and a current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring block may include a spatial neighboring block present in the current picture and a temporal neighboring block present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, a collocated CU (col CU), etc., and the reference picture including the temporal neighboring block may be called a collocated picture (colPic). For example, the inter prediction unit 221 may generate information indicating which candidate is used to construct a motion information candidate list based on neighboring blocks and derive a motion vector and / or a reference picture index of the current block. Inter prediction may be performed based on various prediction modes, and for example, in the case of a skip mode and a merge mode, the inter prediction unit 221 may use motion information of neighboring blocks as motion information of the current block. In the case of the skip mode, unlike the merge mode, a residual signal may not be transmitted.In the case of motion vector prediction (MVP) mode, the motion vector of the current block can be indicated by using the motion vector of a neighboring block as a motion vector predictor and signaling the motion vector difference.

[0049] The prediction unit 220 may generate a prediction signal based on various prediction methods described later. For example, the prediction unit may apply intra prediction or inter prediction for prediction of one block, or may simultaneously apply intra prediction and inter prediction. This may be called combined inter and intra prediction (CIIP). The prediction unit may also be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode may be used for content image / video coding such as games, for example, screen content coding (SCC). IBC basically performs prediction within a current picture, but may be performed similarly to inter prediction in deriving a reference block within the current picture. That is, IBC may use at least one of the inter prediction techniques described in this document. The palette mode may be considered as an example of intra coding or intra prediction. When the palette mode is applied, a sample value within a picture may be signaled based on information regarding a palette table and a palette index.

[0050] The prediction signal generated through the prediction unit (including the inter prediction unit 221 and / or the intra prediction unit 222) may be used to generate a restored signal or may be used to generate a residual signal. The transform unit 232 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally non-linear transform (CNT). Here, the GBT refers to a transform obtained from a graph when the relationship information between pixels is to be expressed as a graph. The CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process may be applied to pixel blocks having the same square size, and may also be applied to non-square variable-size blocks.

[0051] The quantizer 233 quantizes the transform coefficients and transmits the quantized transform coefficients to the entropy encoder 240, which may encode the quantized signal (information on the quantized transform coefficients) and output the quantized signal to a bitstream. The information on the quantized transform coefficients may be referred to as residual information. The quantizer 233 may rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on a coefficient scan order, and may generate information on the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form. The entropy encoder 240 may perform various encoding methods, such as exponential Golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoder 240 may encode information required for video / image restoration (e.g., values ​​of syntax elements, etc.) together with or separately from the quantized transform coefficients. The encoded information (e.g., encoded video / image information) may be transmitted or stored in a network abstraction layer (NAL) unit unit in the form of a bitstream. The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). The video / image information may further include general constraint information. In this document, information and / or syntax elements transmitted / signaled from an encoding device to a decoding device may be included in the video / image information. The video / image information may be encoded through the above-mentioned encoding procedure and included in the bitstream.The bitstream may be transmitted via a network or may be stored in a digital storage medium, where the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as a USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) for transmitting the signal output from the entropy encoding unit 240 and / or a storage unit (not shown) for storing the signal may be configured as an internal / external element of the encoding device 200, or the transmitter may be included in the entropy encoding unit 240.

[0052] The quantized transform coefficients output from the quantizer 233 may be used to generate a prediction signal. For example, a residual signal (residual block or residual sample) may be restored by applying inverse quantization and inverse transform to the quantized transform coefficients via the inverse quantizer 234 and the inverse transformer 235. The adder 155 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter prediction unit 221 or the intra prediction unit 222. When there is no residual for the current block, such as when a skip mode is applied, a predicted block may be used as a reconstructed block. The adder 250 may be referred to as a reconstruction unit or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next current block in the current picture, and may also be used for inter prediction of the next picture after filtering, as described below.

[0053] Meanwhile, luma mapping with chroma scaling (LMCS) may be applied during picture encoding and / or reconstruction.

[0054] The filtering unit 260 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 260 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may store the modified reconstructed picture in the memory 270, specifically, in the DPB of the memory 270. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, etc. The filtering unit 260 may generate various information related to filtering and transmit it to the entropy encoding unit 240, as will be described later in relation to each filtering method. The information related to filtering may be encoded by the entropy encoding unit 240 and output in the form of a bitstream.

[0055] The modified reconstructed picture transmitted to the memory 270 can be used as a reference picture in the inter prediction unit 221. When inter prediction is applied through this, the encoding apparatus can avoid prediction mismatch between the encoding apparatus 100 and the decoding apparatus, and can also improve encoding efficiency.

[0056] The DPB of the memory 270 may store the modified reconstructed picture to be used as a reference picture in the inter prediction unit 221. The memory 270 may store motion information of a block from which motion information in the current picture is derived (or encoded) and / or motion information of a block in an already reconstructed picture. The stored motion information may be transmitted to the inter prediction unit 221 to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory 270 may store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra prediction unit 222.

[0057] FIG. 3 is a diagram illustrating a schematic configuration of a video / image decoding device to which the present disclosure can be applied.

[0058] As shown in FIG. 3, the decoding device 300 may be configured to include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter predictor 331 and an intra predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 321. The entropy decoder 310, the residual processor 320, the predictor 330, the adder 340, and the filter 350 may be configured as one hardware component (e.g., a decoder chipset or processor) according to an embodiment. In addition, the memory 360 may include a decoded picture buffer (DPB) and may be configured as a digital storage medium. The hardware components may further include a memory 360 as an internal / external component.

[0059] If a bitstream including video / image information is input, the decoding device 300 can restore an image corresponding to a process in which the video / image information is processed by the encoding device of FIG. 3. For example, the decoding device 300 can derive a unit / block based on block division related information obtained from the bitstream. The decoding device 300 can perform decoding using a processing unit applied in the encoding device. Thus, the processing unit for decoding can be, for example, a coding unit, and the coding unit can be divided from a coding tree unit or a maximum coding unit according to a quad tree structure, a binary tree structure, and / or a ternary tree structure. One or more transform units can be derived from the coding unit. Then, the restored image signal decoded and output by the decoding device 300 can be reproduced by a reproduction device.

[0060] The decoding device 300 may receive a signal output from the encoding device of FIG. 3 in the form of a bitstream, and the received signal may be decoded via the entropy decoding unit 310. For example, the entropy decoding unit 310 may derive information (e.g., video / image information) required for image restoration (or picture restoration) by parsing the bitstream. The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. The decoding device may further decode pictures based on information on the parameter set and / or the general constraint information. Signaling / received information and / or syntax elements described later in this document may be decoded via the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 310 may decode information in a bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output values ​​of syntax elements required for image restoration and quantized values ​​of transform coefficients related to residuals. More specifically, the CABAC entropy decoding method may receive bins corresponding to each syntax element in the bitstream, determine a context model using syntax element information to be decoded and decode information of neighboring and to-be-decoded blocks or information of symbols / bins decoded in a previous step, predict the occurrence probability of bins according to the determined context model, and perform arithmetic decoding of the bins to generate symbols corresponding to the values ​​of each syntax element. In this case, the CABAC entropy decoding method may update the context model using information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model.Among the information decoded by the entropy decoding unit 310, information related to prediction is provided to a prediction unit (inter prediction unit 332 and intra prediction unit 331), and residual values ​​entropy decoded by the entropy decoding unit 310, i.e., quantized transform coefficients and related parameter information, may be input to the residual processing unit 320. The residual processing unit 320 may derive a residual signal (residual block, residual sample, residual sample array). Also, among the information decoded by the entropy decoding unit 310, information related to filtering may be provided to the filtering unit 350. Meanwhile, a receiving unit (not shown) for receiving a signal output from the encoding device may be further configured as an internal / external element of the decoding device 300, or the receiving unit may be a component of the entropy decoding unit 310. Meanwhile, the decoding device according to this document may be called a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoding unit 310, and the sample decoder may include at least one of the inverse quantization unit 321, the inverse transform unit 322, the addition unit 340, the filtering unit 350, the memory 360, the inter prediction unit 332, and the intra prediction unit 331.

[0061] The inverse quantization unit 321 may inverse quantize the quantized transform coefficients and output the transform coefficients. The inverse quantization unit 321 may rearrange the quantized transform coefficients in a two-dimensional block form. In this case, the rearrangement may be performed based on a coefficient scan order performed in the encoding device. The inverse quantization unit 321 may perform inverse quantization on the quantized transform coefficients using a quantization parameter (e.g., quantization step size information) to obtain transform coefficients.

[0062] The inverse transform unit 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0063] The prediction unit may perform prediction on a current block and generate a predicted block including prediction samples for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoding unit 310, and may determine a specific intra / inter prediction mode.

[0064] The prediction unit 320 may generate a prediction signal based on various prediction methods described later. For example, the prediction unit may apply intra prediction or inter prediction for prediction of one block, and may simultaneously apply intra prediction and inter prediction. This may be called combined inter and intra prediction (CIIP). In addition, the prediction unit may be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode may be used for content image / video coding such as games, for example, screen content coding (SCC). IBC basically performs prediction within a current picture, but may be performed similarly to inter prediction in deriving a reference block within the current picture. That is, IBC may utilize at least one of the inter prediction techniques described in this document. The palette mode may be considered as an example of intra coding or intra prediction. When the palette mode is applied, information regarding a palette table and a palette index may be included in the video / image information and signaled.

[0065] The intra prediction unit 331 may predict a current block by referring to samples in a current picture. The referenced samples may be located in the neighborhood of the current block or may be located away from the current block depending on a prediction mode. In intra prediction, prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit 331 may also determine a prediction mode to be applied to the current block using prediction modes applied to neighboring blocks.

[0066] The inter prediction unit 332 may derive a predicted block for the current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter prediction unit 332 may configure a motion information candidate list based on the neighboring blocks and derive a motion vector and / or a reference picture index for the current block based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and the information on the prediction may include information indicating a mode of inter prediction for the current block.

[0067] The adder 340 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the acquired residual signal to a prediction signal (predicted block, predicted sample array) output from a prediction unit (including an inter prediction unit 332 and / or an intra prediction unit 331). When there is no residual for a current block, such as when a skip mode is applied, the predicted block can be used as a reconstructed block.

[0068] The adder 340 may be referred to as a reconstruction unit or a reconstruction block generator. The generated reconstruction signal may be used for intra prediction of a next block to be processed in a current picture, and may be output after filtering as described below, or may be used for inter prediction of a next picture.

[0069] Meanwhile, LMCS (luma mapping with chroma scaling) can be applied during the picture decoding process.

[0070] The filtering unit 350 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 350 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may transmit the modified reconstructed picture to the memory 360, specifically, to the DPB of the memory 360. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, etc.

[0071] The (modified) reconstructed picture stored in the DPB of the memory 360 may be used as a reference picture in the inter prediction unit 332. The memory 360 may store motion information of a block from which motion information in the current picture is derived (or decoded) and / or motion information of a block in an already reconstructed picture. The stored motion information may be transmitted to the inter prediction unit 260 to be used as motion information of a spatially neighboring block or motion information of a temporally neighboring block. The memory 360 may store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra prediction unit 331.

[0072] In this specification, the embodiments described for the filtering unit 260, inter prediction unit 221, and intra prediction unit 222 of the encoding device 100 can also be applied identically or correspondingly to the filtering unit 350, inter prediction unit 332, and intra prediction unit 331 of the decoding device 300, respectively.

[0073] As described above, prediction is performed to improve compression efficiency when performing video coding. As a result, a predicted block including predicted samples for a current block, which is a block to be coded, can be generated. Here, the predicted block includes predicted samples in a spatial domain (or a pixel domain). The predicted block is derived in the same way in an encoding device and a decoding device, and the encoding device can improve image coding efficiency by signaling information (residual information) about the residual between the original block and the predicted block, which is not an original sample value of the original block, to a decoding device. The decoding device can derive a residual block including residual samples based on the residual information, and can generate a reconstructed block including reconstructed samples by adding the residual block and the predicted block, and can generate a reconstructed picture including the reconstructed block.

[0074] The residual information may be generated through a transform and quantization procedure. For example, the encoding apparatus may derive a residual block between the original block and the predicted block, perform a transform procedure on the residual samples (residual sample array) included in the residual block to derive transform coefficients, and perform a quantization procedure on the transform coefficients to derive quantized transform coefficients, and then signal the related residual information (through a bitstream) to the decoding apparatus. Here, the residual information may include information such as value information, position information, transform technique, transform kernel, and quantization parameter of the quantized transform coefficients. The decoding apparatus may derive a residual sample (or a residual block) by performing an inverse quantization / inverse transform procedure based on the residual information. The decoding apparatus may generate a reconstructed picture based on the predicted block and the residual block. Also, the encoding apparatus may derive a residual block by inverse quantizing / inverse transforming the quantized transform coefficients for reference for inter-prediction of a future picture, and generate a reconstructed picture based on the residual block.

[0075] In one embodiment, a sub-block TMVP flag indicating whether a sub-block based temporal motion vector predictor can be used to control sub-block based motion prediction can be used. The sub-block TMVP flag can be signaled at a sequence parameter set (SPS) level and can control on / off of sub-block based motion prediction. The sub-block TMVP flag can be referred to as sps_sbtmvp_enabled_flag, for example, as shown in Table 1 below.

[0076] Also, to control the affine motion prediction method, an affine flag indicating whether affine prediction can be applied to the current block can be used. The affine flag can be signaled at the SPS level to control on / off of affine prediction. The affine flag can be referred to as sps_affine_enabled_flag, for example, as shown in Table 1 below. If the value of the affine flag is 1, an affine type flag can be additionally signaled to additionally determine whether 6 parameter affine prediction is used.

[0077] An example of the syntax signaled at the SPS level is shown in Table 1 below.

[0078] [Table 1-1]

[0079] [Table 1-2]

[0080] In one embodiment, in a low level coding syntax, as shown in Table 2 below, if the merge flag merge_flag of the current block is 1, a flag (e.g., a merge affine flag) may be signaled to indicate whether an affine merge or a normal merge is applied to the current block based on the condition of the current block (e.g., block size, block shape, etc.) when the affine flag of the SPS is 1. The merge affine flag may be represented as, for example, merge_affine_flag. In one example, if the value of the affine flag signaled at the SPS level is 0 and the value of the merge_flag signaled at the coding unit level is 1, it may be determined that a normal merge is applied to the current block without signaling an additional syntax element.

[0081] An example of syntax signaled at the coding unit level is shown in Table 2 below.

[0082] [Table 2-1]

[0083] [Table 2-2]

[0084] [Table 2-3]

[0085] [Table 2-4]

[0086] Meanwhile, when the high level syntax design of Table 1 and the low level syntax design of Table 2 are applied, if ATMVP is used as an affine merge candidate, design problems, logical problems, conceptual problems, etc. may occur. In one example, if the value of the affine flag signaled at the SPS level is 0 and the value of the sub-block TMVP flag signaled at the SPS level is 1, the ATMVP candidate may not be used as any candidate even though the SPS is signaled to use ATMVP. In addition to the above design problems and logical problems, conceptual problems may exist. ATMVP is a sub-block (SubPu in one example) based motion prediction method, and in order to distinguish between non-sub-block based (non SubPu in one example) motion prediction candidates and sub-block based motion prediction candidates in normal merge, it is used as a candidate for affine merge mode that performs prediction on a sub-block basis, thereby having the purpose of distinguishing whether the merge of the current block is a sub-block merge or a non-sub-block merge. However, despite this objective, in the low-level syntax design according to Table 2, the sub-block ATMVP is controlled depending on whether or not affine merging is available.

[0087] To address the design, logical, and conceptual issues discussed above, in one embodiment, a high-level and / or low-level syntax design may be provided based on at least one of Tables 3 through 11 below.

[0088] In one embodiment, a flag for controlling sub-block based motion prediction may be signaled at an SPS level. The flag for controlling sub-block based motion prediction may be expressed as, for example, sps_subpumvp_enabled_flag, and may be used to determine whether the sub-block based motion prediction is on / off. When the value of the sps_subpumvp_enabled_flag is 1, affine_enabled_flag and sbtmvp_enabled_flag may be signaled as shown in Table 3 below.

[0089] [Table 3]

[0090] When using the SPS level syntax design of Table 3, the availability of affine prediction and ATMVP can be expressed as shown in Table 4 below. In Table 4 below, 1 indicates that the method is available, and 0 indicates that the method is not available.

[0091] [Table 4]

[0092] In one embodiment, a high level syntax design may be provided for controlling whether or not affine prediction and ATMVP are both available based on sps_subpumvp_enabled_flag. In one example, according to this embodiment, if the value of sps_subpumvp_enabled_flag is 1, it may be determined that both affine prediction and ATMVP are available. The high level syntax design according to this embodiment may be as shown in Table 5 below.

[0093] [Table 5]

[0094] In one embodiment, whether or not affine prediction and ATMVP are both enabled is controlled based on sps_subpumvp_enabled_flag included in the high level syntax according to Table 5, but in order to control whether or not ATMVP is enabled in detail for each slice, a method of using slice_subpumvp_enabled_flag in slice header syntax may be provided. The syntax at the slice header level according to this embodiment may be, for example, as shown in Table 6 below.

[0095] [Table 6]

[0096] In one embodiment, when the affine prediction method is not used and sps_sbtmvp_enabled_flag is 1, a method may be provided in which merge_affine_flag is signaled but affine candidates are not configured as candidates, and only ATMVP is configured as a candidate. An example of a low-level syntax for expressing this embodiment may be as shown in Table 7 below.

[0097] [Table 7]

[0098] In Table 7, if the value of sps_affine_enabled_flag is 1 or the value of sps_sbtmvp_enabled_flag is 1, it may be determined that the merge affine flag merge_affine_flag, which indicates whether the merge affine mode is applied, is decoded.

[0099] In one example, it may be determined to decode a merge sub-block flag merge_subblock_flag indicating whether a merge sub-block mode is applied when the value of sps_affine_enabled_flag is 1 or the value of sps_sbtmvp_enabled_flag is 1. In the merge sub-block mode, merge candidates may be determined on a sub-block basis.

[0100] In Table 7, if the width (cbWidth) and height (cbHeight) of the current block are each 8 or greater, and the value of sps_affine_enabled_flag is 1 or the value of sps_sbtmvp_enabled_flag is 1, it can be determined that the merge affine flag merge_affine_flag is decoded.

[0101] In one example, if the maximum number of merging candidates of the sub-blocks of the current block is greater than 0, it may be determined to decode the pre-determined merge mode flag.

[0102] In one example, if the affine flag has a value of 1 or the sub-block TMVP flag has a value of 1, the maximum number of merging candidates of the sub-block of the current block may be greater than 0.

[0103] In one example, whether to decode the pre-determined merge mode flag may be determined based on whether if(MaxNumSubblockMergeCand>0&&cbWidth>=8&&cbHeight>=8) is satisfied, where MaxNumSubblockMergeCand represents the maximum number of merge candidates for the subblock, cbWidth represents the width of the current block, and cbHeight represents the height of the current block.

[0104] In Table 7, if the value of sps_affine_enabled_flag is 0 and the value of sps_sbtmvp_enabled_flag is 1, merge_affine_idx is not signaled and can be inferred to 0. According to the embodiment of Table 7, the availability of affine prediction and ATMVP can be expressed as shown in Table 8 below.

[0105] [Table 8]

[0106] In one embodiment, a method may be provided for controlling ATMVP to be used as a normal merging candidate when the affine prediction method is not used and the value of sps_sbtmvp_enabled_flag is 1. According to this embodiment, whether or not affine prediction and ATMVP can be used may be expressed as shown in Table 9 below.

[0107] [Table 9]

[0108] In one embodiment, a method may be provided for designing a high level syntax such that sps_sbtmvp_enabled_flag is signaled only if the value of affine_enabled_flag is 1. This may take into consideration the structure of a low level coding tool designed such that ATMVP is used as an affine merge candidate and cannot be used if the value of sps_affine_enabled_flag is 0. An example of the high level syntax according to this embodiment is shown in Table 10 below.

[0109] [Table 10]

[0110] When using the SPS level syntax design of Table 10, the availability of affine prediction and ATMVP can be expressed as shown in Table 11 below.

[0111] [Table 11]

[0112] FIG. 4 is a flowchart showing the operation of an encoding device according to an embodiment, and FIG. 5 is a block diagram showing the configuration of an encoding device according to an embodiment.

[0113] The encoding apparatus according to Figures 4 and 5 may perform operations corresponding to those of the decoding apparatus according to Figures 6 and 7. Therefore, the operations of the decoding apparatus described below in Figures 6 and 7 may be similarly applied to the encoding apparatus according to Figures 4 and 5.

[0114] Each step disclosed in Fig. 4 may be performed by the encoding device 200 disclosed in Fig. 2. More specifically, S400 and S410 may be performed by the prediction unit 220 disclosed in Fig. 2, and S420 may be performed by the entropy encoding unit 240 disclosed in Fig. 2. Furthermore, the operations according to S400 to S420 are based on a part of the contents described above in Fig. 3. Therefore, the description of specific contents that overlap with the contents described above in Figs. 2 and 3 will be omitted or simplified.

[0115] As shown in Fig. 5, the encoding device according to an embodiment may include a prediction unit 220 and an entropy encoding unit 240. However, in some cases, not all of the components shown in Fig. 5 may be essential components of the encoding device, and the encoding device may be realized with more or less components than the components shown in Fig. 5.

[0116] In the encoding device according to an embodiment, the prediction unit 220 and the entropy encoding unit 240 may be implemented on separate chips, or at least two of the components may be implemented on a single chip.

[0117] An encoding device according to an embodiment may determine whether affine prediction is applicable to a current block and whether a temporal motion vector predictor based on a sub-block of the current block may be used (S400). More specifically, a prediction unit 220 of the encoding device may determine whether affine prediction is applicable to a current block and whether a temporal motion vector predictor based on a sub-block of the current block may be used.

[0118] An encoding device according to an embodiment may determine whether to encode a pre-determined merge mode flag indicating whether to apply a pre-determined merge mode to the current block based on the determination of whether the affine prediction is applicable to the current block and whether the temporal motion vector predictor based on the sub-block of the current block can be used (S410). More specifically, a prediction unit 220 of the encoding device may determine whether to encode a pre-determined merge mode flag indicating whether to apply a pre-determined merge mode to the current block based on the determination of whether the affine prediction is applicable to the current block and whether the temporal motion vector predictor based on the sub-block of the current block can be used.

[0119] In one example, the predetermined merge mode may be a merge affine mode or a merge sub-block mode, and the predetermined merge mode flag may be a merge affine flag or a merge sub-block flag, where the merge affine flag may be represented as merge_affine_flag, and the merge sub-block flag may be represented as merge_subblock_flag.

[0120] An encoding device according to an embodiment may encode an affine flag indicating whether the affine prediction can be applied to the current block, a sub-block TMVP flag indicating whether the temporal motion vector predictor based on the sub-block of the current block can be used, and the determined merge mode flag based on the determination of whether the determined merge mode flag is to be encoded (S420). More specifically, an entropy encoding unit 240 of the encoding device may encode an affine flag indicating whether the affine prediction can be applied to the current block, a sub-block TMVP flag indicating whether the temporal motion vector predictor based on the sub-block of the current block can be used, and the determined merge mode flag based on the determination of whether the determined merge mode flag is to be encoded.

[0121] In one embodiment, if the value of the affine flag is 1 or the value of the sub-block TMVP flag is 1, it may be determined to encode the pre-determined merge mode flag.

[0122] In one embodiment, if the width and height of the current block are each greater than or equal to 8 and a first condition is satisfied in which the value of the affine flag is 1, or a second condition is satisfied in which the value of the sub-block TMVP flag is 1, it may be determined that the pre-determined merge mode flag is encoded.

[0123] In one embodiment, whether to encode the determined merge mode flag may be determined based on Equation 1 below.

[0124]

number

[0125] In Equation 1, sps_affine_enabled_flag represents the affine flag, cbWidth represents the width of the current block, cbHeight represents the height of the current block, and sps_sbtmvp_enabled_flag represents the sub-block TMVP flag.

[0126] In one embodiment, the predetermined merge mode flag may be a merge affine flag indicating whether an affine merge mode is applied to the current block or a merge sub-block flag indicating whether a merge mode is applied to the sub-block unit of the current block.

[0127] In one embodiment, if the maximum number of merging candidates of the sub-blocks of the current block is greater than 0, it may be determined to encode the determined merge mode flag.

[0128] In an embodiment, if the value of the affine flag is 1 or the value of the sub-block TMVP flag is 1, the maximum number of merging candidates of the sub-block of the current block may be greater than 0.

[0129] In one embodiment, whether to encode the determined merge mode flag may be determined based on Equation 2 below.

[0130]

number

[0131] In Equation 2, MaxNumSubblockMergeCand may represent the maximum number of merge candidates for the subblock, cbWidth may represent the width of the current block, and cbHeight may represent the height of the current block.

[0132] According to the encoding device and the operating method of the encoding device of Figures 4 and 5, the encoding device determines whether affine prediction can be applied to a current block and whether a temporal motion vector predictor based on a sub-block of the current block can be used (S400), determines whether to encode a predetermined merge mode flag indicating whether a predetermined merge mode is to be applied to the current block based on the determination of whether the affine prediction can be applied to the current block and whether the temporal motion vector predictor based on the sub-block of the current block can be used (S410), and encodes an affine flag indicating whether the affine prediction can be applied to the current block, a sub-block TMVP flag indicating whether the temporal motion vector predictor based on the sub-block of the current block can be used, and the predetermined merge mode flag based on the determination of whether to encode the predetermined merge mode flag (S420), and may be characterized in that if the value of the affine flag is 1 or the value of the sub-block TMVP flag is 1, it is determined that the predetermined merge mode flag is to be encoded. That is, based on the affine flag and the sub-block TMVP flag, it is determined whether to decode a pre-determined merge mode flag, which indicates whether a pre-determined merge mode is applied to the current block, thereby improving image coding efficiency.

[0133] FIG. 6 is a flowchart showing the operation of a decoding device according to an embodiment, and FIG. 7 is a block diagram showing the configuration of a decoding device according to an embodiment.

[0134] Each step disclosed in FIG. 6 may be performed by the decoding device 300 disclosed in FIG. 3. More specifically, S600 and S610 may be performed by the entropy decoding unit 310 disclosed in FIG. 3, S620 may be performed by the prediction unit 330 disclosed in FIG. 3, and S630 may be performed by the addition unit 340 disclosed in FIG. 3. Furthermore, the operations according to S600 to S630 are based on some of the contents described above in FIG. 3. Therefore, the detailed description that overlaps with the contents described above in FIG. 3 will be omitted or simplified.

[0135] As shown in Fig. 7, a decoding device according to an embodiment may include an entropy decoding unit 310, a prediction unit 330, and an addition unit 340. However, in some cases, all of the components shown in Fig. 7 may not be essential components of the decoding device, and the decoding device may be realized with more or less components than the components shown in Fig. 7.

[0136] In one embodiment of the decoding device, the entropy decoding unit 310, the prediction unit 330, and the addition unit 340 may each be implemented on a separate chip, or at least two or more components may be implemented on a single chip.

[0137] A decoding device according to an embodiment can decode, based on a bitstream, an affine flag indicating whether affine prediction can be applied to a current block and a sub-block TMVP flag indicating whether a temporal motion vector predictor based on a sub-block of the current block can be used (S600). More specifically, the entropy decoding unit 310 of the decoding device can decode, based on a bitstream, an affine flag indicating whether affine prediction can be applied to a current block and a sub-block TMVP flag indicating whether a temporal motion vector predictor based on a sub-block of the current block can be used.

[0138] In one example, the affine flag may be represented as sps_affine_enabled_flag, and the sub-block TMVP flag may be represented as sps_sbtmvp_enabled_flag. The sub-block TMVP flag may also be referred to as a sub-PU TMVP flag in some cases.

[0139] In one example, the affine flag and the sub-block TMVP flag can be signaled at the SPS level.

[0140] A decoding device according to an embodiment may determine whether to decode a pre-determined merge mode flag indicating whether to apply a pre-determined merge mode to the current block based on the decoded affine flag and the decoded sub-block TMVP flag (S610). More specifically, an entropy decoding unit 310 of the decoding device may determine whether to decode a pre-determined merge mode flag indicating whether to apply a pre-determined merge mode to the current block based on the decoded affine flag and the decoded sub-block TMVP flag.

[0141] In one example, the predetermined merge mode may be a merge affine mode or a merge sub-block mode, and the predetermined merge mode flag may be a merge affine flag or a merge sub-block flag, where the merge affine flag may be represented as merge_affine_flag, and the merge sub-block flag may be represented as merge_subblock_flag.

[0142] According to an embodiment, the decoding device may derive a prediction sample for the current block based on the determination of whether to decode the determined merge mode flag (S620). More specifically, the prediction unit 330 of the decoding device may derive a prediction sample for the current block based on the determination of whether to decode the determined merge mode flag.

[0143] A decoding device according to one embodiment can derive a prediction mode to be applied to the current block based on the decision as to whether to decode the pre-determined merge mode flag, and can derive a prediction sample for the current block based on the derived prediction mode.

[0144] According to an embodiment, the decoding device may generate reconstructed samples for the current block based on the predicted samples for the current block (S630). More specifically, the adder 340 of the decoding device may generate reconstructed samples for the current block based on the predicted samples for the current block.

[0145] In one embodiment, if the value of the affine flag is 1 or the value of the sub-block TMVP flag is 1, it may be determined to decode the pre-determined merge mode flag.

[0146] In one example, if the value of sps_affine_enabled_flag is 1 or the value of sps_sbtmvp_enabled_flag is 1, it may be determined that the pre-determined merge mode flag is to be decoded.

[0147] In another example, if the value of sps_affine_enabled_flag is 1 or the value of sps_sbtmvp_enabled_flag is 1, it may be determined that the merge affine flag merge_affine_flag is decoded.

[0148] In yet another example, if the value of sps_affine_enabled_flag is 1 or the value of sps_sbtmvp_enabled_flag is 1, it may be determined that the merge sub-block flag merge_subblock_flag is decoded.

[0149] In one embodiment, if the width and height of the current block are each greater than or equal to 8 and a first condition is satisfied in which the value of the affine flag is 1, or a second condition is satisfied in which the value of the sub-block TMVP flag is 1, it may be determined that the pre-determined merge mode flag is decoded.

[0150] In one embodiment, whether to decode the pre-determined merge mode flag may be determined based on Equation 3 below.

[0151]

number

[0152] In Equation 3, sps_affine_enabled_flag represents the affine flag, cbWidth represents the width of the current block, cbHeight represents the height of the current block, and sps_sbtmvp_enabled_flag represents the sub-block TMVP flag.

[0153] In one embodiment, if the maximum number of merging candidates of the sub-blocks of the current block is greater than 0, it may be determined to decode the pre-determined merge mode flag.

[0154] In an embodiment, if the affine flag has a value of 1 or the sub-block TMVP flag has a value of 1, the maximum number of merging candidates of the sub-block of the current block may be greater than 0.

[0155] In one embodiment, whether to decode the pre-determined merge mode flag may be determined based on Equation 4 below.

[0156]

number

[0157] In Equation 4, MaxNumSubblockMergeCand may represent the maximum number of merge candidates for the subblock, cbWidth may represent the width of the current block, and cbHeight may represent the height of the current block.

[0158] According to the decoding device and the operating method of the decoding device disclosed in Figures 6 and 7, the decoding device decodes an affine flag indicating whether affine prediction can be applied to a current block and a sub-block TMVP flag indicating whether a temporal motion vector predictor based on a sub-block of the current block can be used based on a bitstream (S600), determines whether to decode a predetermined merge mode flag indicating whether a predetermined merge mode is to be applied to the current block based on the decoded affine flag and the decoded sub-block TMVP flag (S610), derives a prediction sample for the current block based on the decision on whether to decode the predetermined merge mode flag (S620), and generates a reconstructed sample for the current block based on the prediction sample for the current block (S630), and may be characterized in that if the value of the affine flag is 1 or the value of the sub-block TMVP flag is 1, it is determined that the predetermined merge mode flag is to be decoded. That is, based on the affine flag and the sub-block TMVP flag, it is determined whether to decode a pre-determined merge mode flag, which indicates whether a pre-determined merge mode is applied to the current block, thereby improving image coding efficiency.

[0159] In the above-described embodiments, the method is described based on a flowchart as a series of steps or blocks, but the present disclosure is not limited to the order of steps, and certain steps may occur in a different order or simultaneously with different steps than those described above. Also, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, and other steps may be included, or one or more steps of the flowchart may be deleted without affecting the scope of the present disclosure.

[0160] The above-described method according to the present disclosure can be realized in software form, and the encoding device and / or decoding device according to the present disclosure can be included in an image processing device such as a TV, a computer, a smartphone, a set-top box, or a display device.

[0161] When the embodiments and the like in the present disclosure are implemented in software, the above-described methods can be implemented in modules (processes, functions, etc.) that perform the above-described functions. The modules can be stored in a memory and executed by a processor. The memory can be inside or outside the processor and can be coupled to the processor in various well-known ways. The processor can include an application-specific integrated circuit (ASIC), other chipsets, logic circuits, and / or data processing devices. The memory can include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium, and / or other storage devices. That is, the embodiments and the like described in the present disclosure can be implemented and performed on a processor, microprocessor, controller, or chip. For example, the functional units illustrated in each drawing can be implemented and performed on a computer, processor, microprocessor, controller, or chip. In this case, information (e.g., information on instructions) or algorithms for implementation can be stored in a digital storage medium.

[0162] In addition, the decoding device and the encoding device to which the present disclosure is applied may be included in a multimedia broadcast transmitting / receiving device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video interactive device, a real-time communication device such as a video communication, a mobile streaming device, a storage medium, a camcorder, a custom video (VoD) service providing device, an over-the-top video (OTT) device, an internet streaming service providing device, a three-dimensional (3D) video device, a virtual reality (VR) device, an argument reality (AR) device, an image telephone video device, a transportation means terminal (e.g., a vehicle (including an autonomous vehicle) terminal, an airplane terminal, a ship terminal, etc.), and a medical video device, and may be used to process a video signal or a data signal. For example, an over-the-top video (OTT) device may include a game console, a Blu-ray player, an internet-connected TV, a home theater system, a smartphone, a tablet PC, a digital video recorder (DVR), etc.

[0163] In addition, the processing method to which the present disclosure is applied may be produced in the form of a program executed by a computer and may be stored in a computer-readable recording medium. Multimedia data having a data structure according to the present disclosure may also be stored in a computer-readable recording medium. The computer-readable recording medium includes any kind of storage device and distributed storage device in which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray Disc (BD), a Universal Serial Bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The computer-readable recording medium may also include a medium realized in the form of a carrier wave (e.g., transmission via the Internet). Also, a bit stream generated by the encoding method may be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network.

[0164] Furthermore, the embodiments of the present disclosure may be implemented in a computer program product by program code, which may be executed by a computer in accordance with the embodiments of the present disclosure. The program code may be stored on a computer readable carrier.

[0165] FIG. 8 illustrates an example of a content streaming system to which the disclosure of this document may be applied.

[0166] As shown in FIG. 8, a content streaming system to which the present disclosure is applied can broadly include an encoding server, a streaming server, a web server, a media repository, a user device, and a multimedia input device.

[0167] The encoding server compresses content input from a multimedia input device such as a smartphone, camera, camcorder, etc. into digital data to generate a bitstream and transmits the bitstream to the streaming server. As another example, if a multimedia input device such as a smartphone, camera, camcorder, etc. directly generates a bitstream, the encoding server may be omitted.

[0168] The bitstream may be generated by an encoding method or a bitstream generation method to which the present disclosure is applied, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0169] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server acts as a medium to inform the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server transmits multimedia data to the user. In this case, the content streaming system may include a separate control server, and in this case, the control server controls commands / responses between devices in the content streaming system.

[0170] The streaming server may receive content from a media repository and / or an encoding server. For example, when the content is received from the encoding server, the content may be received in real time. In this case, the streaming server may store the bitstream for a certain period of time in order to provide a smooth streaming service.

[0171] Examples of the user devices include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glass, head mounted displays (HMDs)), digital TVs, desktop computers, digital signage, etc.

[0172] Each server in the content streaming system can be operated as a distributed server, in which case data received by each server can be processed in a distributed manner.

Claims

1. An image decoding method performed by a decoding device, receiving image information including affine enable flag information and sub-block temporal motion vector prediction enable flag information; receiving specific flag information relating to whether a sub-block based specific merge mode is applied to a current block; determining whether to receive a specific merge index for the sub-block based specific merge mode based on the specific flag information, the affine enable flag information, and the sub-block temporal motion vector prediction enable flag information; deriving a prediction sample for the current block by applying inter prediction to the current block based on the specific flag information; generating reconstructed samples based on the predicted samples; The step of receiving the specific flag information includes: determining whether to receive the specific flag information based on at least one of the affine enable flag information and the sub-block temporal motion vector prediction enable flag information; receiving the specific flag information based on a result of determining whether to receive the specific flag information; determining that the specific merge index is not received based on a case where a value of the specific flag information is equal to 1, a value of the affine enable flag information is equal to 0, and the sub-block temporal motion vector prediction enable flag information is equal to 1; The image decoding method, wherein the sub-block temporal motion vector prediction enable flag information is configured in a sequence parameter set.

2. An image encoding method performed by an encoding device, comprising: deriving affine enable flag information and sub-block temporal motion vector prediction enable flag information; deriving a predicted sample for a current block by applying inter prediction to the current block; deriving specific flag information relating to whether a sub-block based specific merge mode is applied to the current block; determining whether to signal a specific merge index for the sub-block based specific merge mode based on the specific flag information, the affine enable flag information, and the sub-block temporal motion vector prediction enable flag information; encoding image information including at least one of the affine enable flag information, the sub-block temporal motion vector prediction enable flag information, the specific flag information, and the specific merge index; The step of deriving the specific flag information includes: determining whether to signal the specific flag information based on at least one of the affine enable flag information and the sub-block temporal motion vector prediction enable flag information; and deriving the specific flag information based on a result of determining whether to signal the specific flag information; It is determined that the specific merge index is not signaled based on a case where the value of the specific flag information is equal to 1, the value of the affine enable flag information is equal to 0, and the sub-block temporal motion vector prediction enable flag information is equal to 1; The image encoding method, wherein the sub-block temporal motion vector prediction enable flag information is configured in a sequence parameter set.

3. 1. A method of transmitting data for an image, comprising: Obtaining a bitstream, the bitstream is generated by: deriving affine enable flag information and sub-block temporal motion vector prediction enable flag information; deriving a prediction sample for the current block by applying inter prediction to the current block; deriving specific flag information related to whether a sub-block based specific merge mode is applied to the current block; determining whether to signal a specific merge index for the sub-block based specific merge mode based on the specific flag information, the affine enable flag information, and the sub-block temporal motion vector prediction enable flag information; and generating the bitstream by encoding image information including at least one of the affine enable flag information, the sub-block temporal motion vector prediction enable flag information, the specific flag information, and the specific merge index. transmitting said data including said bitstream; The deriving of the specific flag information includes: determining whether to signal the specific flag information based on at least one of the affine enable flag information and the sub-block temporal motion vector prediction enable flag information; deriving the specific flag information based on a result of determining whether to signal the specific flag information; It is determined that the specific merge index is not signaled based on a case where the value of the specific flag information is equal to 1, the value of the affine enable flag information is equal to 0, and the sub-block temporal motion vector prediction enable flag information is equal to 1; The method, wherein the sub-block temporal motion vector prediction enable flag information is configured in a sequence parameter set.

Images