Image encoding / decoding method and apparatus, and recording medium on which bit stream is stored

Abstract

An image decoding / encoding method and apparatus according to the present disclosure may: derive a time vector of a current block based on a motion vector of one candidate in a candidate group including a plurality of candidates; determining a collocated block of the current block in the collocated picture based on the time vector; deriving a motion vector of the current block by the sub-block unit based on the motion vector of the collocated block; and performing inter prediction on the current block based on the motion vector of the current block.

Description

Technical Field

[0001] The present disclosure relates to an image encoding / decoding method and apparatus and a recording medium storing a bit stream. Background Art

[0002] Recently, demands for high-resolution and high-quality images such as HD (High Definition) images and UHD (Ultra High Definition) images have been increasing in various application fields, and therefore, efficient image compression technology is being discussed.

[0003] There are various technologies, such as inter-frame prediction technology that uses video compression technology to predict pixel values ​​included in the current picture from pictures before or after the current picture, intra-frame prediction technology that predicts pixel values ​​included in the current picture by using pixel information in the current picture, entropy coding technology that assigns short symbols to values ​​with high occurrence frequency and assigns long symbols to values ​​with low occurrence frequency, etc., and these image compression technologies can be used to effectively compress image data and transmit or store it. Summary of the invention

[0004] Technical issues

[0005] The present disclosure is to provide a method and apparatus for performing inter prediction by using a subblock based temporal motion vector predictor (SbTMVP).

[0006] The present disclosure is to provide a method and apparatus for considering more diverse candidates when deriving a sub-block based temporal motion vector predictor.

[0007] Technical Solution

[0008] According to the image decoding method and device of the present disclosure, a time vector of a current block can be derived based on a motion vector of one candidate in a candidate group including multiple candidates; based on the time vector, a collocated block of the current block in a collocated picture is determined; based on the motion vector of the collocated block, a motion vector of the current block in a sub-block unit is derived; and based on the motion vector of the current block, inter-frame prediction for the current block is performed.

[0009] According to the image decoding method and apparatus of the present disclosure, the candidate group may include adjacent spatial neighboring blocks of the current block and non-adjacent spatial neighboring blocks of the current block as candidates.

[0010] In the image decoding method and device according to the present disclosure, the non-adjacent spatially adjacent blocks may include: a first block including samples of a value obtained by multiplying the height of the current block by 2 separated from an upper left sample adjacent to the upper left corner of the current block within a left sample line adjacent to the current block; and a second block including samples of a value obtained by multiplying the width of the current block by 2 separated from an upper left sample within a top sample line adjacent to the current block.

[0011] In the image decoding method and apparatus according to the present disclosure, one candidate may be selected from a plurality of candidates included in a candidate group based on template matching.

[0012] In the image decoding method and apparatus according to the present disclosure, the cost of template matching may be calculated based on the sum of absolute differences (SAD) or mean squared absolute differences (MRSAD) between a template area of ​​a current block and a template area of ​​a block specified by a plurality of candidate motion vectors included in a candidate group.

[0013] In the image decoding method and apparatus according to the present disclosure, the cost of template matching may be calculated based on the sum of absolute differences (SAD) or mean squared absolute differences (MRSAD) between a template area of ​​a current block and a template area of ​​a subblock or block specified by a motion vector of a block derived based on a plurality of candidates included in a candidate group.

[0014] The image decoding method and apparatus according to the present disclosure may configure a candidate group including a plurality of candidates based on neighboring blocks of a current block.

[0015] In the image decoding method and apparatus according to the present disclosure, the candidate group may be configured by adding neighboring blocks at specific positions to the candidate group in a predefined order.

[0016] The image decoding method and apparatus according to the present disclosure may confirm whether a motion vector of a neighboring block at a specific position overlaps with a motion vector of a candidate previously included in a candidate group.

[0017] In the image decoding method and apparatus according to the present disclosure, whether there is overlap may be determined based on whether a difference between a motion vector of a candidate previously included in a candidate group and a motion vector of a neighboring block at a specific position is less than a predefined threshold.

[0018] In the image decoding method and apparatus according to the present disclosure, a collocated picture may be determined based on a syntax element signaled through at least one syntax of a picture header or a slice header.

[0019] In the image decoding method and apparatus according to the present disclosure, a syntax element may be signaled separately from a syntax element indicating a collocated picture for a temporal motion vector predictor.

[0020] According to the image encoding method and device of the present disclosure, a time vector of a current block can be determined based on a motion vector of one candidate in a candidate group including multiple candidates, a collocated block of the current block in a collocated picture can be determined based on the time vector, a motion vector of the current block in a sub-block unit can be determined based on the motion vector of the collocated block, and inter-frame prediction for the current block can be performed based on the motion vector of the current block.

[0021] In the image encoding method and apparatus according to the present disclosure, the candidate group may include adjacent spatial neighboring blocks of the current block and non-adjacent spatial neighboring blocks of the current block as candidates.

[0022] In the image decoding method and device according to the present disclosure, the non-adjacent spatially adjacent blocks may include: a first block including samples of a value obtained by multiplying the height of the current block by 2 separated from an upper left sample adjacent to the upper left corner of the current block within a left sample line adjacent to the current block; and a second block including samples of a value obtained by multiplying the width of the current block by 2 separated from an upper left sample within a top sample line adjacent to the current block.

[0023] In the image decoding method and apparatus according to the present disclosure, one candidate may be selected from a plurality of candidates included in a candidate group based on template matching.

[0024] In the image decoding method and apparatus according to the present disclosure, the cost of template matching may be calculated based on the sum of absolute differences (SAD) or mean squared absolute differences (MRSAD) between a template area of ​​a current block and a template area of ​​a block specified by a plurality of candidate motion vectors included in a candidate group.

[0025] In the image decoding method and apparatus according to the present disclosure, the cost of template matching may be calculated based on the sum of absolute differences (SAD) or mean squared absolute differences (MRSAD) between a template area of ​​a current block and a template area of ​​a subblock or block specified by a motion vector of a block derived based on a plurality of candidates included in a candidate group.

[0026] The image decoding method and apparatus according to the present disclosure may configure a candidate group including a plurality of candidates based on neighboring blocks of a current block.

[0027] In the image decoding method and apparatus according to the present disclosure, the candidate group may be configured by adding neighboring blocks at specific positions to the candidate group in a predefined order.

[0028] The image decoding method and apparatus according to the present disclosure may confirm whether a motion vector of a neighboring block at a specific position overlaps with a motion vector of a candidate previously included in a candidate group.

[0029] In the image decoding method and apparatus according to the present disclosure, whether there is overlap may be determined based on whether a difference between a motion vector of a candidate previously included in a candidate group and a motion vector of a neighboring block at a specific position is less than a predefined threshold.

[0030] In the image decoding method and apparatus according to the present disclosure, a collocated picture may be determined based on a syntax element signaled through at least one syntax of a picture header or a slice header.

[0031] In the image decoding method and apparatus according to the present disclosure, a syntax element may be signaled separately from a syntax element indicating a collocated picture for a temporal motion vector predictor.

[0032] A computer-readable digital storage medium is provided, which stores encoded video / image information, resulting in execution of an image decoding method due to a decoding device according to the present disclosure.

[0033] A computer-readable digital storage medium storing video / image information generated according to an image encoding method according to the present disclosure is provided.

[0034] A method and apparatus for transmitting video / image information generated according to an image encoding method according to the present disclosure are provided.

[0035] Technical Effects

[0036] When deriving a sub-block based temporal motion vector predictor, the present disclosure may consider more various candidates for prediction, increase the accuracy of prediction and improve compression performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A video / image compilation system according to the present disclosure is shown.

[0038] Figure 2 A rough block diagram of an encoding device to which an embodiment of the present disclosure can be applied and which performs encoding of a video / image signal is shown.

[0039] Figure 3 A rough block diagram of a decoding device to which an embodiment of the present disclosure can be applied and which performs decoding of a video / image signal is shown.

[0040] Figure 4 An inter-frame prediction method performed by the decoding device 300 according to an embodiment of the present disclosure is shown.

[0041] Figure 5 is a flow chart illustrating a process for deriving a sub-block based motion vector predictor to which embodiments of the present disclosure may be applied.

[0042] Figure 6 is a flowchart illustrating a process for deriving a sub-block based motion vector predictor according to an embodiment of the present disclosure.

[0043] Figure 7 is a diagram illustrating neighboring blocks to which embodiments of the present disclosure may be applied.

[0044] Figure 8 is a diagram for describing a method for checking overlap between motion vectors according to an embodiment of the present disclosure.

[0045] Fig. 9 is a flowchart illustrating a method for configuring a merge candidate list according to an embodiment of the present disclosure.

[0046] Fig.10 is a diagram illustrating a method for deriving temporal merging candidates according to an embodiment of the present disclosure.

[0047] Fig.11 is a diagram illustrating a method for deriving motion information from a reference block according to an embodiment of the present disclosure.

[0048] Fig.12 is a diagram illustrating a method for deriving motion information from a reference block according to an embodiment of the present disclosure.

[0049] Fig.13 is a diagram illustrating a method for deriving motion information from a reference block according to an embodiment of the present disclosure.

[0050] Fig.14 is a diagram illustrating a method for deriving motion information from a reference block according to an embodiment of the present disclosure.

[0051] Fig.15 A schematic configuration of the inter-frame predictor 332 performing the intra-frame prediction method according to the present disclosure is shown.

[0052] Fig.16 An inter-frame prediction method performed by the encoding apparatus 200 as an embodiment of the present disclosure is shown.

[0053] Fig.17 A schematic configuration of the inter-frame predictor 221 performing the inter-frame prediction method according to the present disclosure is shown.

[0054] Fig.18 An example of a content streaming system to which an embodiment of the present disclosure can be applied is shown. DETAILED DESCRIPTION

[0055] Because the present disclosure can make various changes and has several embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, it is not intended to limit the present disclosure to specific embodiments, and it should be understood to include all changes, equivalents and substitutes included in the spirit and technical scope of the present disclosure. While describing each of the accompanying drawings, similar reference numerals are used for similar components.

[0056] Terms such as first, second, etc. can be used to describe various components, but components should not be limited by these terms. These terms are only used to distinguish one component from other components. For example, without departing from the scope of the present disclosure, a first component can be referred to as a second component, and similarly, a second component can also be referred to as a first component. Terms and / or combinations of any one or more related statement items in a plurality of related statement items are included.

[0057] When a component is referred to as being "connected" or "linked" to another component, it should be understood that it can be directly connected or linked to another component, but another component may also exist in between. On the other hand, when a component is referred to as being "directly connected" or "directly linked" to another component, it should be understood that another component does not exist in between.

[0058] The terms used in this application are only used to describe specific embodiments and are not intended to limit the present disclosure. Unless the context clearly indicates otherwise, singular expressions include plural expressions. In this application, it should be understood that terms such as "including" or "having" are intended to designate the existence of features, numbers, steps, operations, components, parts or combinations thereof described in the specification, but do not exclude the possibility of the existence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof in advance.

[0059] The present disclosure relates to video / image coding. For example, the methods / embodiments disclosed herein may be applied to methods disclosed in the Universal Video Coding (VVC) standard. In addition, the methods / embodiments disclosed herein may be applied to methods disclosed in the Basic Video Coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the second generation audio video coding standard (AVS2), or the next generation video / image coding standard (e.g., H.267 or H.268, etc.).

[0060] This specification proposes various embodiments of video / image coding, and unless otherwise stated, these embodiments may be performed in combination with each other.

[0061] Here, video may refer to a collection of a series of images over time. A picture generally refers to a unit representing an image within a specific time period, and a slice / tile is a unit that forms a part of a picture in coding. A slice / tile may include at least one coding tree unit (CTU). A picture may consist of at least one slice / tile. A tile is a rectangular area consisting of multiple CTUs within a specific tile column and a specific tile row of a picture. A tile column is a rectangular area of ​​a CTU having the same height as the picture and a width assigned by the syntax requirements of the picture parameter set. A tile row is a rectangular area of ​​a CTU having a height assigned by the picture parameter set and the same width as the width of the picture. The CTU within a tile may be arranged continuously according to a CTU raster scan, and the tiles within a picture may be arranged continuously according to a raster scan of the tile. A slice may include an integer number of complete tiles or an integer number of continuous complete CTU rows within a tile of a picture that may be exclusively included in a single NAL unit. At the same time, a picture may be divided into at least two sub-pictures. A sub-picture may be a rectangular area of ​​at least one slice within a picture.

[0062] Pixel, pixel or picture element may refer to the smallest unit constituting a picture (or image). In addition, "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luminance component, or only a pixel / pixel value of a chrominance component.

[0063] A unit may represent a basic unit of image processing. A unit may include at least one of a specific region of a picture and information associated with the corresponding region. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, a unit may be used interchangeably with terms such as a block or region. In general, an MxN block may include a set (or array) of transform coefficients or samples (or sample arrays) consisting of M columns and N rows.

[0064] Here, "A or B" may refer to "only A", "only B", or "both A and B". In other words, herein, "A or B" may be interpreted as "A and / or B". For example, herein, "A, B or C" may refer to "only A", "only B", "only C", or "any combination of A, B, and C".

[0065] As used herein, a slash ( / ) or a comma may mean "and / or". For example, "A / B" may mean "A and / or B". Thus, "A / B" may mean "only A", "only B", or "both A and B". For example, "A, B, C" may mean "A, B, or C".

[0066] Here, "at least one of A and B" may refer to "only A", "only B", or "both A and B". In addition, herein, expressions such as "at least one of A or B" or "at least one of A and / or B" can be interpreted in the same manner as "at least one of A and B".

[0067] In addition, herein, “at least one of A, B, and C” may refer to “only A”, “only B”, “only C”, or “any combination of A, B, and C”. In addition, “at least one of A, B, or C” or “at least one of A, B and / or C” may refer to “at least one of A, B, and C”.

[0068] In addition, the brackets used herein may refer to "for example". Specifically, when the indication is "prediction (intra-frame prediction)", "intra-frame prediction" may be proposed as an example of "prediction". In other words, the "prediction" here is not limited to "intra-frame prediction", and "intra-frame prediction" may be proposed as an example of "prediction". In addition, even when the indication is "prediction (ie, intra-frame prediction)", "intra-frame prediction" may be proposed as an example of "prediction".

[0069] Here, technical features described individually in one drawing may be implemented individually or simultaneously.

[0070] Figure 1 A video / image compilation system according to the present disclosure is shown.

[0071] refer to Figure 1 , a video / image coding system may include a first device (source device) and a second device (receiving device).

[0072] The source device may send the encoded video / image information or data to the receiving device in the form of a file or stream transmission through a digital storage medium or a network. The source device may include a video source, an encoding device, and a sending unit. The receiving device may include a receiving unit, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be composed of a separate device or an external component.

[0073] The video source may obtain the video / image through the process of capturing, synthesizing or generating the video / image. The video source may include a device for capturing the video / image and a device for generating the video / image. The device for capturing the video / image may include at least one camera, a video / image archive including previously captured videos / images, etc. The device for generating the video / image may include a computer, a tablet computer, a smart phone, etc., and may (electronically) generate the video / image. For example, a virtual video / image may be generated by a computer, etc., and in this case, the process of capturing the video / image may be replaced by the process of generating the relevant data.

[0074] The encoding device can encode the input video / image. The encoding device can perform a series of processes such as prediction, transformation, quantization, etc. for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0075] The sending unit may send the encoded video / image information or data output in the form of a bit stream to the receiving unit of the receiving device in the form of a file or stream transmission through a digital storage medium or a network. The digital storage medium may include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The sending unit may include an element for generating a media file in a predetermined file format and may include an element for transmission through a broadcast / communication network. The receiving unit may receive / extract a bit stream and send it to a decoding device.

[0076] The decoding device may decode the video / image by performing a series of processes corresponding to the operations of the encoding device, such as dequantization, inverse transformation, prediction, etc.

[0077] The renderer may render the decoded video / image. The rendered video / image may be displayed through a display unit.

[0078] Figure 2 A rough block diagram of an encoding device to which an embodiment of the present disclosure can be applied and which performs encoding of a video / image signal is shown.

[0079] refer to Figure 2, the encoding device 200 may be composed of an image segmenter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. According to an embodiment, the above-mentioned image segmenter 210, the predictor 220, the residual processor 230, the entropy encoder 240, the adder 250, and the filter 260 may be configured by at least one hardware component (e.g., an encoder chipset or processor). In addition, the memory 270 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include a memory 270 as an internal / external component.

[0080] The image divider 210 may divide the input image (or picture, frame) input to the encoding device 200 into at least one processing unit. As an example, the processing unit may be referred to as a coding unit (CU). In this case, the coding unit may be recursively divided from a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree binary tree ternary tree (QTBTTT) structure.

[0081] For example, one coding unit may be segmented into a plurality of coding units having a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary structure. In this case, for example, the quadtree structure may be applied first, and the binary tree structure and / or the ternary structure may be applied later. Alternatively, the binary tree structure may be applied before the quadtree structure. The coding process according to this specification may be performed based on a final coding unit that is no longer segmented. In this case, based on coding efficiency according to image characteristics, etc., the maximum coding unit may be directly used as the final coding unit, or if necessary, the coding unit may be recursively segmented into coding units of a deeper depth, and the coding unit with the best size may be used as the final coding unit. Here, the coding process may include processes such as prediction, transformation, and reconstruction described later.

[0082] As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be divided or partitioned from the above-mentioned final coding unit, respectively. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient.

[0083] In some cases, a unit may be used interchangeably with terms such as a block or region. In general, an MxN block may represent a set of transform coefficients or samples consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luma component, or only a pixel / pixel value of a chroma component. A sample may be used as a term to make a picture (or image) correspond to a pixel or a picture element.

[0084] The encoding device 200 may subtract the prediction signal (prediction block, prediction sample array) output from the inter predictor 221 or the intra predictor 222 from the input image signal (original block, original sample array) to generate a residual signal (residual signal, residual sample array), and the generated residual signal is transmitted to the transformer 232. In this case, a unit that subtracts the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) within the encoding device 200 may be referred to as a subtractor 231.

[0085] The predictor 220 may perform prediction on a block to be processed (hereinafter referred to as a current block) and generate a predicted block including a prediction sample for the current block. The predictor 220 may determine whether intra prediction or inter prediction is applied in units of a current block or CU. The predictor 220 may generate various information about the prediction, such as prediction mode information, and send it to the entropy encoder 240, as described later in the description of each prediction mode. The information about the prediction may be encoded in the entropy encoder 240 and output in the form of a bitstream.

[0086] The intra-frame predictor 222 can predict the current block by referring to the samples in the current picture. Depending on the prediction mode, the referenced sample can be located near the current block or can be located at a certain distance away from the current block. In intra-frame prediction, the prediction mode may include at least one non-directional mode and multiple directional modes. The non-directional mode may include at least one of the DC mode or the plane mode. Depending on the detail level of the prediction direction, the directional mode may include 33 directional modes or 65 directional modes. However, this is only an example, and more or less directional modes may be used depending on the configuration. The intra-frame predictor 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0087] The inter-frame predictor 221 may derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter-frame predictor 221 may configure a motion information candidate list based on the neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, and for example, for skip mode and merge mode, the inter predictor 221 may use motion information of neighboring blocks as motion information of the current block. For skip mode, unlike merge mode, a residual signal may not be transmitted. For motion vector prediction (MVP) mode, motion vectors of surrounding blocks are used as motion vector predictors, and motion vector differences are signaled to indicate the motion vector of the current block.

[0088] The predictor 220 may generate a prediction signal based on various prediction methods described later. For example, the predictor may not only apply intra prediction or inter prediction to predict a block, but may also apply intra prediction and inter prediction at the same time. It may be referred to as a combined inter and intra prediction (CIIP) mode. In addition, the predictor may be based on an intra block copy (IBC) prediction mode or may be based on a palette mode for prediction of a block. The IBC prediction mode or the palette mode may be used for content image / video coding of games, such as screen content coding (SCC), etc. IBC basically performs prediction within the current picture, but it may be performed similarly to inter prediction because it derives a reference block within the current picture. In other words, IBC may use at least one of the inter prediction techniques described herein. The palette mode may be considered an example of intra coding or intra prediction. When the palette mode is applied, the sample values ​​within the picture may be signaled based on information about the palette table and the palette index. The prediction signal generated by the predictor 220 may be used to generate a reconstructed signal or a residual signal.

[0089] The transformer 232 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loève transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, GBT refers to a transform obtained from a graph when relationship information between pixels is expressed as a graph. CNT refers to a transform obtained based on generating a prediction signal using all previously reconstructed pixels. In addition, the transform process may be applied to square pixel blocks of the same size or may be applied to non-square blocks of variable size.

[0090] The quantizer 233 may quantize the transform coefficients and send them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantizer 233 may rearrange the quantized transform coefficients in the block form into a one-dimensional vector form based on the coefficient scanning order, and may generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0091] The entropy encoder 240 may perform various encoding methods such as Exponential Golomb, Context Adaptive Variable Length Coding (CAVLC), Context Adaptive Binary Arithmetic Coding (CABAC), etc. The entropy encoder 240 may encode information necessary for video / video image reconstruction (e.g., values ​​of syntax elements, etc.) in addition to transform coefficients quantized together or individually.

[0092] The encoded information (e.g., encoded video / image information) can be transmitted or stored in units of network abstraction layer (NAL) units in the form of a bitstream. The video / image information may further include information about various parameter sets such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. Here, information and / or syntax elements transmitted / signaled from the encoding device to the decoding device may be included in the video / image information. The video / image information may be encoded and included in the bitstream through the above-mentioned encoding process. The bitstream may be transmitted through a network or may be stored in a digital storage medium. Here, the network may include a broadcast network and / or a communication network, etc., and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmission unit (not shown) for transmission and / or a storage unit (not shown) for storing a signal output from the entropy encoder 240 may be configured as an internal / external element of the encoding device 200, or the transmission unit may also be included in the entropy encoder 240.

[0093] The quantized transform coefficients output from the quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients through the dequantizer 234 and the inverse transformer 235. The adder 250 can add the reconstructed residual signal to the prediction signal output from the inter-frame predictor 221 or the intra-frame predictor 222 to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). When there is no residual of the block to be processed, such as when the skip mode is applied, the prediction block can be used as a reconstructed block. The adder 250 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture, and can also be used for inter-frame prediction of the next picture through filtering described later. At the same time, luminance mapping with chroma scaling (LMCS) can be applied in the picture encoding and / or reconstruction process.

[0094] The filter 260 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 260 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and the modified reconstructed picture can be stored in the memory 270, specifically in the DPB of the memory 270. Various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filter 260 can generate various information about filtering and send it to the entropy encoder 240. The information about filtering can be encoded in the entropy encoder 240 and output in the form of a bit stream.

[0095] The modified reconstructed picture transmitted to the memory 270 may be used as a reference picture in the inter predictor 221. When inter prediction is applied therethrough, the encoding apparatus can avoid prediction mismatch in the encoding apparatus 200 and the decoding apparatus, and can also improve encoding efficiency.

[0096] The DPB of the memory 270 may store the modified reconstructed picture to be used as a reference picture in the inter-frame predictor 221. The memory 270 may store the motion information of the block from which the motion information in the current picture is derived (or encoded) and / or the motion information of the block in the pre-reconstructed picture. The stored motion information may be sent to the inter-frame predictor 221 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 270 may store the reconstructed samples of the reconstructed block in the current picture and send them to the intra-frame predictor 222.

[0097] Figure 3 A rough block diagram of a decoding device to which an embodiment of the present disclosure can be applied and which performs decoding of a video / image signal is shown.

[0098] refer to Figure 3 , the decoding device 300 may be configured by including an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 331 and an intra-frame predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 321.

[0099] According to an embodiment, the above-mentioned entropy decoder 310, residual processor 320, predictor 330, adder 340 and filter 350 may be configured by one hardware component (e.g., decoder chipset or processor). In addition, the memory 360 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory 360 as an internal / external component.

[0100] When a bit stream including video / image information is input, the decoding device 300 may respond to the Figure 2 The image is reconstructed by a process of processing video / image information in an encoding device. For example, the decoding device 300 can derive a unit / block based on relevant information of block segmentation obtained from a bitstream. The decoding device 300 can perform decoding by using a processing unit applied in the encoding device. Therefore, the processing unit of decoding can be a coding unit, and the coding unit can be divided from a coding tree unit or a maximum coding unit according to a quadtree structure, a binary tree structure and / or a ternary tree structure. At least one transform unit can be derived from the coding unit. And, the reconstructed image signal decoded and output by the decoding device 300 can be played by a playback device.

[0101] The decoding device 300 may receive the bit stream from Figure 2 The received signal can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) necessary for image reconstruction (or picture reconstruction). The video / image information may further include information about various parameter sets such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. The decoding device may further decode the picture based on the information about the parameter set and / or the general constraint information. The signaled / received information and / or the syntax elements described later in this document may be decoded and obtained from the bitstream by a decoding process. For example, the entropy decoder 310 may decode the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, CABAC, etc., and output the values ​​of the syntax elements necessary for image reconstruction and the quantized values ​​of the transform coefficients of the residual. In more detail, the CABAC entropy decoding method can receive a bin corresponding to each syntax element from a bitstream, determine a context model by using information of syntax elements to be decoded, decoding information of surrounding blocks and blocks to be decoded, or information of symbols / bins decoded in the previous step, perform arithmetic decoding on bins by predicting the probability of occurrence of bins according to the determined context model, and generate symbols corresponding to the values ​​of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method can update the context model by using information about decoded symbols / bins of the context model for the next symbol / bin. Among the information decoded in the entropy decoder 310, information about prediction is provided to the predictor (inter-frame predictor 332 and intra-frame predictor 331), and the residual value, that is, the quantized transform coefficient and related parameter information, which is entropy decoded in the entropy decoder 310, can be input to the residual processor 320. The residual processor 320 can derive a residual signal (residual block, residual sample, residual sample array). In addition, information about filtering among the information decoded in the entropy decoder 310 can be provided to the filter 350. Meanwhile, a receiving unit (not shown) that receives a signal output from the encoding device may be further configured as an internal / external element of the decoding device 300 or the receiving unit may be a component of the entropy decoder 310 .

[0102] Meanwhile, the decoding device according to this specification may be referred to as a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.

[0103] The dequantizer 321 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 321 may rearrange the quantized transform coefficients into a two-dimensional block form. In this case, the rearrangement may be performed based on the coefficient scanning order performed in the encoding device. The dequantizer 321 may dequantize the quantized transform coefficients by using a quantization parameter (e.g., quantization step size information) and obtain the transform coefficients.

[0104] The inverse transformer 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0105] The predictor 320 may perform prediction on the current block and generate a prediction block including prediction samples for the current block. The predictor 320 may determine whether to apply intra prediction or inter prediction to the current block based on the information on prediction output from the entropy decoder 310, and determine a specific intra / inter prediction mode.

[0106] The predictor 320 can generate a prediction signal based on various prediction methods described later. For example, the predictor 320 can not only apply intra prediction or inter prediction to predict a block, but also apply intra prediction and inter prediction at the same time. It can be called a combined inter and intra prediction (CIIP) mode. In addition, the predictor can be based on an intra block copy (IBC) prediction mode or can be based on a palette mode for prediction of a block. The IBC prediction mode or the palette mode can be used for content image / video coding of games, such as screen content coding (SCC), etc. IBC basically performs prediction within the current picture, but it can be performed similarly to inter prediction because it derives a reference block within the current picture. In other words, IBC can use at least one of the inter prediction techniques described herein. The palette mode can be considered as an example of intra coding or intra prediction. When the palette mode is applied, information about the palette table and the palette index can be included in the video / image information and sent with a signal.

[0107] The intra-frame predictor 331 can predict the current block by referring to samples within the current picture. Depending on the prediction mode, the referenced sample can be located near the current block or can be located at a certain distance away from the current block. In intra-frame prediction, the prediction mode may include at least one non-directional mode and multiple directional modes. The intra-frame predictor 331 can determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

[0108] The inter-frame predictor 332 may derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter-frame predictor 332 may configure a motion information candidate list based on the neighboring blocks, and derive a motion vector and / or a reference picture index for the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes, and information about the prediction may include information indicating an inter-frame prediction mode for the current block.

[0109] The adder 340 may add the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including the inter-frame predictor 332 and / or the intra-frame predictor 331) to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). When there is no residual of the block to be processed, such as when the skip mode is applied, the prediction block may be used as the reconstructed block.

[0110] The adder 340 may be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next block to be processed in the current picture, may be output through filtering described later, or may be used for inter prediction of the next picture. At the same time, luminance mapping with chroma scaling (LMCS) may be applied during picture decoding.

[0111] The filter 350 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 350 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and send the modified reconstructed picture to the memory 360, specifically the DPB of the memory 360. The various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0112] The (modified) reconstructed picture stored in the DPB of the memory 360 can be used as a reference picture in the inter-frame prediction unit 332. The memory 360 can be derived from the motion information in its current picture (or decoded) The motion information of the block and / or the motion information of the block in the pre-reconstructed picture. The stored motion information can be sent to the inter-frame predictor 260 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 360 can store the reconstructed samples of the reconstructed block in the current picture and send them to the intra-frame predictor 331.

[0113] Here, the embodiments described in the filter 260, the inter-frame predictor 221, and the intra-frame predictor 222 of the encoding device 200 may also be equally or correspondingly applied to the filter 350, the inter-frame predictor 332, and the intra-frame predictor 331 of the decoding device 300, respectively.

[0114] Figure 4 An inter-frame prediction method performed by the decoding device 300 according to an embodiment of the present disclosure is shown.

[0115] In an embodiment of the present disclosure, when performing inter-frame prediction by using a sub-block-based temporal motion vector predictor (SbTMVP), a method for improving prediction accuracy by considering more diverse candidates is proposed. In the present disclosure, the sub-block-based temporal motion vector predictor refers to a predictor derived based on derived motion information by deriving motion information of temporally adjacent blocks in a sub-block unit, and of course, the name is not limited thereto. In the present disclosure, for the convenience of description, the sub-block-based temporal motion vector predictor may be abbreviated as SbTMVP.

[0116] refer to Figure 4 , the decoding device may derive a time vector of the current block based on a motion vector of one candidate in a candidate group including a plurality of candidates S400.

[0117] Unlike TMVP, which derives a motion vector predictor (MVP) from motion information of a block at a predefined position (i.e., the lower right position or the center position of the current block) within a predetermined reference picture (or collocated picture, colPic), SbTMVP in sub-block merge mode uses motion information of a collocated block (or col block) derived from motion information of a neighboring block (e.g., the left neighboring block of the current block) as the MVP of the current block and derives the MVP in a sub-block unit. In an embodiment of the present disclosure, since the accuracy of the motion information of the neighboring blocks is high and various candidates are considered, the performance of SbTMVP can be improved, so a method for this is proposed.

[0118] In other words, an improved method in the process of deriving SbTMVP is proposed, which takes into account various time vectors for specifying collocated blocks. The proposed method can be applied not only to the process for deriving sub-block-based MVP, but also to the process for deriving MVP in the MERGE / AMVP process in substantially the same manner.

[0119] According to an embodiment of the present disclosure, in order to improve the performance of the existing SbTMVP that uses the motion information of the collocated block derived from the motion information of the left neighboring block as the MVP, the best MVP can be derived by considering various motion information derived from multiple neighboring blocks. As an example, the candidate group may include the adjacent spatial neighboring blocks of the current block and the non-adjacent spatial neighboring blocks of the current block as candidates. A detailed description will be made with reference to the following figure.

[0120] The decoding device may determine (or specify) a collocated block of the current block in the collocated picture based on the derived time vector S410. As an example, the collocated picture may be identically defined for the encoding device and the decoding device. Alternatively, as an example, the collocated picture may be signaled by a higher level syntax. The collocated block of the current block may be determined by a time vector within the collocated picture.

[0121] The decoding device may derive the motion vector of the current block in subblock units based on the motion vector of the collocated block S420. In other words, the decoding device may derive (or determine) the motion vector of the subblock within the collocated block specified by the time vector as the motion vector of the corresponding subblock within the current block.

[0122] The decoding device may perform inter prediction for the current block based on the motion vector of the current block S430. The decoding device may generate a prediction block of the current block by performing inter prediction based on motion information derived in sub-block units within the current block. As an embodiment, the prediction block of the current block may be generated in sub-block units according to the motion information derived in the sub-block units.

[0123] Figure 5 is a flow chart illustrating a process for deriving a sub-block based motion vector predictor to which embodiments of the present disclosure may be applied.

[0124] exist Figure 5 In , it is assumed that only the motion vector of the left neighboring block (ie, the block adjacent to the left side of the lower left sample A1 of the current block) is used as the temporal vector of the existing SbTMVP.

[0125] refer to Figure 5, when using SbTMVP, the collocated block can be derived by using the motion information of the left neighboring block as the time vector. First, it can be confirmed whether the motion information of the left neighboring block is available. When the motion information of the left neighboring block is available, the time vector can configure the motion information of the left neighboring block as a time vector. When the motion information of the left neighboring block is not available, the time vector can be configured as a zero vector.

[0126] The collocated block may be specified by a temporal vector in a collocated picture. When a default motion vector at a center position of the collocated block is available, each motion information in a sub-block unit may be stored as an MVP. In other words, when a default motion vector is available, motion information in a sub-block unit may be derived. The availability of a default motion vector may be a condition for deriving motion information in a sub-block unit.

[0127] The motion information of the subblock that is unavailable in the step for deriving the motion information in the subblock unit may be replaced with a default motion vector. For example, it may be determined that the collocated block is available in the inter mode and unavailable in the intra block copy (IBC) mode.

[0128] Figure 6 is a flowchart illustrating a process for deriving a sub-block based motion vector predictor according to an embodiment of the present disclosure.

[0129] refer to Figure 6 In order to consider more diverse motion information according to the embodiments of the present disclosure, the above can be modified and applied Figure 5 The SbTMVP process described in Figure 6 shown.

[0130] According to an embodiment of the present disclosure, a temporal vector may be derived by using neighboring blocks at various predefined positions. In other words, neighboring blocks at various predefined positions may be used as SbTMVP candidates.

[0131] As an embodiment, the temporal vector may be derived by using a plurality of neighboring blocks {A1, B1, B0, A0, B2, D0, D1}. The temporal vector may be derived using a motion vector of a candidate specified in a candidate group including a plurality of candidates. In other words, the candidate group may be {A1, B1, B0, A0, B2, D0, D1} including a plurality of neighboring blocks.

[0132] refer to Figure 6 , when the motion vector at the center position of the collocated block (i.e., the default motion vector) is available in the candidate group, it can be considered as a SbTMVP candidate. The process for deriving the temporal vector is the same as Figure 6 As shown or mentioned above Figure 5The same as described in, and repeated descriptions related thereto are omitted. This process may be repeated until MAX_NUM candidates are available or all candidates are checked. Here, MAX_NUM may represent the maximum number of SbTMVP candidates. MAX_NUM may be a predefined value. Alternatively, MAX_NUM may be a value explicitly signaled or a value implicitly derived based on coding information.

[0133] When MAX_NUM candidates are not filled despite all checks being performed on all designated neighboring blocks, a zero vector may be considered as a temporal vector. Afterwards, the candidate with the lowest cost may be determined based on the template matching (or template matching cost) of the SbTMVP candidates obtained through this process. The MVP in each sub-block unit may be finally derived by using the determined candidates.

[0134] In the present disclosure, the candidate group for deriving the time vector may represent a candidate group including a plurality of predefined neighboring blocks as candidates. Figure 6 In the illustrated embodiment, the candidate group for deriving the time vector may represent a candidate group including a maximum number (MAX_NUM) or less of candidates whose availability is confirmed among a plurality of predefined neighboring blocks.

[0135] Figure 7 is a diagram illustrating neighboring blocks to which embodiments of the present disclosure may be applied.

[0136] Figure 7 Various reference blocks that can be considered for deriving a temporal vector are shown. In other words, blocks adjacent to the current block and blocks not adjacent to the current block can be used when deriving a temporal vector for specifying a collocated block. In the present disclosure, they may be referred to as adjacent blocks and non-adjacent blocks, respectively.

[0137] As an embodiment, the collocated block can be derived by using candidate blocks adjacent to the current block. As an example of an adjacent block, consider Figure 7 {A1, B1, B0, A0, B2, A2, B3} shown in . This is an example, and the order of consideration can be changed, and some of the candidate blocks listed can be added or omitted. For example, the adjacent blocks are not limited to this example, and all blocks adjacent to the current block can be its target. The adjacent blocks may include blocks at a specified position among the blocks adjacent to the current block. As an embodiment, it can be determined by scanning and checking all 4x4 blocks adjacent to the left / top.

[0138] Specifically, the block at the left position may be scanned from the bottom to the top. In other words, the scanning may be performed from the lower left neighboring block of the current block to the upper left neighboring block of the current block. The block at the top position may be scanned from the right to the left. In other words, the scanning may be performed from the upper right neighboring block of the current block to the upper left neighboring block of the current block.

[0139] Alternatively, the block at the left position may be scanned from the top to the bottom. In other words, the scanning may be performed from the upper left neighboring block of the current block to the lower left neighboring block of the current block. The block at the top position may be scanned from the left to the right. In other words, the scanning may be performed from the upper left neighboring block of the current block to the upper right neighboring block of the current block. However, this is an example and may be changed and applied.

[0140] In addition, as an embodiment, the number of blocks on the left / top side can be defined in the process for checking the left and top blocks. As an example, when scanning the left block in a specified order, if there are K available blocks, the scanning of the left block can be terminated and the scanning of the top block can be performed. When there are L available top blocks, the scanning of the top block can be terminated. In this case, the values ​​of K and L can be predefined values, or can be values ​​variably determined according to the shape of the block.

[0141] According to an embodiment of the present disclosure, a collocated block can be derived by using a candidate that is not adjacent to the current block, so as to consider various reference blocks. According to the video encoding / decoding order, only the left block and the top block of the current block are valid, so in the process of deriving MVP, only the motion vectors of the adjacent left block and the top block are considered.

[0142] To overcome this shortcoming, the TMVP technique refers to the motion information at the bottom and right positions of the current block. In this case, for the collocated block, the lower right position or the center position within the collocated picture is considered. In an embodiment of the present disclosure, similar to the purpose of giving priority to the lower right position in the above-mentioned TMVP, various SbTMVP candidates can be used.

[0143] As an embodiment, the non-adjacent spatially neighboring block may include a first block including samples separated from an upper left sample adjacent to an upper left corner of the current block within a left sample line adjacent to the current block by a value obtained by multiplying the height of the current block by 2; and a second block including samples separated from an upper left sample within a top sample line adjacent to the current block by a value obtained by multiplying the width of the current block by 2.

[0144] As an example, non-adjacent blocks may be used as Figure 7 The blocks of D0 and D1 are shown to facilitate consideration of movement at the bottom and / or right positions. This is an example, and the positions of the D0 and D1 blocks may be applied as one or more of the candidates according to the following equation 1, respectively.

[0145] [Equation 1]

[0146]

[0147] In Equation 1, in this case, W and H represent the width and height of the current block, respectively. In Equation 1, W and H can be applied by replacing them with specified values. For example, W and H can be defined as values ​​of 2, 4, 8, 16, 32.

[0148] In addition, non-adjacent block candidates may be determined according to the shape of the current block. As an example, for W>=H, non-adjacent blocks may be determined as in Equation 2 below.

[0149] [Equation 2]

[0150]

[0151] Referring to Equation 2, for W>=H, it may be adjusted to a position different from the position of the lower left block derived by Equation 1. In addition, as an example, for W<H, the non-adjacent blocks may be determined as in Equation 3 below.

[0152] [Equation 3]

[0153]

[0154] Referring to Equation 3, for W < H, it can be adjusted to a position different from the position of the upper right block derived by Equation 1.

[0155] In addition, zero vectors may be considered to consider various reference blocks. While the existing SbTMVP replaces the left block with a zero vector when the left block is unavailable, the proposed method may use a zero vector when the number of candidates is not filled, without using it as a replacement candidate for another candidate.

[0156] According to an embodiment of the present disclosure, by using motion vectors of neighboring blocks, motion vectors of non-neighboring blocks, and a zero vector, up to MAX_NUM candidates may be considered as SbTMVP candidates, increasing the accuracy of SbTMVP prediction and improving compression performance.

[0157] Hereinafter, a method for considering more diverse candidates through effective overlap checking is described when deriving SbTMVP by using multiple motion vectors (or temporal vectors) as in the above-described embodiments. The embodiments described below may apply overlap checking to the above-described multiple candidates.

[0158] For example, the overlap can be checked by using the time vectors derived from the adjacent blocks. In other words, it can be confirmed whether there is overlap between the time vectors.

[0159] Alternatively, for example, overlap may be checked by using a motion vector (DefaultMV) at the center position of the collocated block derived using a time vector.

[0160] Alternatively, for example, overlap can be checked by using a motion vector in each sub-block unit of a collocated block derived using a time vector. In this case, overlap check can be performed on motion vectors of all sub-blocks in the collocated block. Alternatively, overlap check can be performed on sub-blocks at specific positions in the collocated block.

[0161] The overlap check method can be applied to the motion vectors of adjacent blocks, the motion vectors of non-adjacent blocks, and the zero vector. In addition, in order to reduce the complexity of the overlap check, it can be applied only to some candidates. In other words, the overlap check can be performed between predefined specific candidates.

[0162] As an example, because Figure 7 The A0 block and the A1 block in are very adjacent, so the overlap check may be applied, and because the A0 block and the B0 block are not adjacent to each other, it can be inferred that they have different motion information and the overlap check may not be applied. Alternatively, the overlap check may be applied only to blocks adjacent to each other, and the overlap check may not be applied to non-adjacent blocks and zero vectors.

[0163] Figure 8 is a diagram for describing a method for checking overlap between motion vectors according to an embodiment of the present disclosure.

[0164] refer to Figure 8 , the overlap check can be determined by whether there is the same reference picture and the same motion vector, and as an embodiment, when the difference between the motion vectors is less than a certain threshold, it can be determined as an overlap candidate.

[0165] Specifically, Figure 8 As shown in , the motion information in the collocated picture can be stored in a specific unit. For example, the motion information in the collocated picture can be stored as motion information in 8x8 units. Therefore, when the overlap check is performed based on the time vector, if the difference between each candidate motion vector is less than a specific value, it may exist in the same unit and the motion information may overlap.

[0166] Therefore, according to an embodiment, more diverse candidates can be configured by determining whether there is overlap based on a threshold. As an example, the threshold may be predefined, or may be defined as 1 pixel, 1 / 4 pixel, 1 / 16 pixel, etc.

[0167] Hereinafter, a method for determining the best candidate among a plurality of SbTMVP candidates derived according to the above method is described.

[0168] Inter prediction may include a sub-block merge mode among general merge modes. The sub-block merge mode may include an SbTMVP mode and an affine merge mode. When the sub-block merge mode is used, a sub-block merge candidate list including N candidates may be configured. The index of a candidate in the candidate list may be sent by a signal. In an embodiment, the candidate list may include one SbTMVP candidate and N-1 affine merge candidates.

[0169] According to an embodiment of the present disclosure, although the number of candidates for SbTMVP increases, the performance of affine merging candidates can be maintained by finally selecting one SbTMVP candidate and configuring the candidate list as before. To this end, a template matching method can be applied. In other words, the candidate with the minimum template matching cost among multiple SbTMVP candidates can be selected as the SbTMVP.

[0170] The template matching cost may be calculated by using at least one of the following methods: The methods may be varied and applied by considering the trade-off between accuracy and complexity.

[0171] -The Sum of Absolute Differences (SAD) or Mean-Reduced Sum of Absolute Differences (MRSAD) in the template area can be calculated by using the temporal vector derived based on each adjacent / non-adjacent / zero vector. Because it is based on the template similarity between the current block and the collocated block derived from the temporal vector, a temporal vector with high reliability can be found. Its advantage is that it can be calculated without increasing the complexity compared to calculating the DefaultMv and sub-block MV for each candidate.

[0172] -SAD or MRSAD in the template area between the current block and the reference block indicated by the MV (DefaultMV) at the center position of the collocated block derived by using the time vector can be calculated. Because it uses the motion vector of a representative reference block for practical applications (i.e., used as a prediction sample), it is advantageous in that the accuracy as MVP can be improved compared to when the time vector is used.

[0173] -The SAD or MRSAD in the template area can be calculated by using the MV in each sub-block unit within the collocated block derived using the time vector. Because it uses the motion vector actually applied in each sub-block unit, it has the highest accuracy as an MVP, but it may relatively increase the complexity. In this case, because all sub-blocks may not have a template area according to the position of the sub-block in the current block, in an embodiment, the template matching cost may be calculated only for the sub-blocks adjacent to the left and top of the current block.

[0174] When calculating the template matching cost in the CU unit (ie, the current block), the template area may be determined as WxN or NxH. W and H represent the width and height of the CU, respectively. By taking into account the complexity of the calculation and the size of the block, N may be determined as 1, 2, 4, etc. When calculating the template matching cost in the sub-block unit, for the template area, all or part of WxM and MxH may be used. By taking into account the complexity of the calculation and the size of the block, M may be determined as 1, 2, 4, etc.

[0175] In addition, the above method can be changed and applied as follows. When the number of sub-block merge candidates increases and because the performance is improved when considering multiple candidates, multiple candidates among the SbTMVP candidates configured in the above embodiment can be used as the final sub-block merge candidates. As an example, the sub-block merge candidate list can be configured with two SbTMVP candidates. In this case, among the SbTMVP candidates derived as two SbTMVP candidates, a candidate with the smallest template matching cost and a candidate with the second smallest template matching cost can be selected.

[0176] Specifically, the signaling method for SbTMVP may be changed and applied as follows: For example, the sub-block merging candidate list may be configured as shown in Table 1 below.

[0177] [Table 1]

[0178]

[0179] Referring to Table 1, as in the example of Method 1, a certain number (eg, 2) of SbTMVP candidates may be added to the sub-block merge candidate list, and then, an affine merge candidate may be added.

[0180] In addition, a flag indicating whether the sub-block merge candidate list includes an SbTMVP candidate may be signaled. In other words, as in the example of method 2, when in sub-block merge mode, the configuration of the sub-block merge candidate list may be changed by signaling a flag (SbTMVPFlag) indicating whether the SbTMVP exists. When SbTMVPFlag is 1, the SbTMVP candidate list may be configured, and when SbTMVPFlag is 0, the affine candidate list may be configured.

[0181] Because method 1 in Table 1 can add an affine merge candidate to the sub-block merge candidate list when there is no or only one SbTMVP candidate, waste in signaling bit efficiency can be prevented. At the same time, the advantage of method 2 is that it can consider various candidates for the SbTMVP mode while maintaining the same bit usage for the affine candidate, instead of adding the SbTMVPFlag. In an embodiment, method 2 can be configured to generate as many SbTMVP candidates as the maximum number of candidates for the SbTMVP candidate list, so as to prevent unnecessary transmission of the SbTMVPFlag.

[0182] Fig. 9 is a flowchart illustrating a method for configuring a merge candidate list according to an embodiment of the present disclosure.

[0183] SbTMVP operates as a partial tool of the sub-block merge mode among the general merge mode because it predicts motion information in sub-block units. Unlike TMVP, SbTMVP does not obtain motion information at the determined position, so it can perform more accurate motion prediction. Therefore, according to an embodiment of the present disclosure, a method for applying SbTMVP is proposed, which is the same as the above-mentioned SbTMVP, but derives motion information in the coding block unit instead of the sub-block unit as one of the candidates for the conventional merge mode. In the present disclosure, the SbTMVP in the sub-block unit that does not include the derivation method is referred to as Moved-TMVP (M-TMVP). The name is not limited to this, and M-TMVP may be referred to as TMVP for short.

[0184] refer to Fig. 9 , a merge candidate list according to a conventional merge mode may be configured. The merge candidate list may include spatial candidates, temporal candidates, non-adjacent candidates, a history-based motion vector predictor (HMVP), and a zero candidate. As an example, spatial candidates, temporal candidates, non-adjacent candidates, HMVP, and zero candidates may be inserted into the merge candidate list in the order of them.

[0185] In this case, the temporal candidates may include the M-TMVP according to this embodiment and the existing TMVP. As an embodiment, modifications may be performed such as replacing the TMVP with the M-TMVP or a candidate with a higher compression efficiency than the TMVP and selecting the M-TMVP. In this case, template matching may be used to determine a candidate with a high compression efficiency. In addition, the temporal candidates may be selected in accordance with Fig. 9 The M-TMVP is added to the merge candidate list in different orders.

[0186] Above Figure 5 and Figure 6The method described in the above can be applied to derive M-TMVP. As described above, M-TMVP can derive motion information in block units instead of sub-block units. In this case, the block unit can be a coding block unit. Figure 5 and Figure 6 In the method described in , the step of deriving the motion vector in the sub-block unit can be omitted, or replaced by the step of deriving the motion vector in the block unit. Figure 5 and Figure 6 In the method described in , operations excluding the step of deriving a motion vector in a sub-block unit may also be similarly applied to M-TMVP, and repeated descriptions related thereto are omitted.

[0187] Fig.10 is a diagram illustrating a method for deriving temporal merging candidates according to an embodiment of the present disclosure.

[0188] refer to Fig.10 , for M-TMVP that does not derive motion information in sub-block units, it can be changed to collect various motion information different from the existing motion vector. Fig.10 Assume the following case, the motion vector of the left neighboring block adjacent to the left side of the lower left sample of the current block is used as the time vector to specify the collocated block, but the above Figure 6 and 7 Neighboring blocks at various positions described in and the left neighboring block may be used as temporal vector candidates.

[0189] When using M-TMVP, in order to use more diverse motion information for prediction, the position of the collocated block may be changed and applied as follows. In other words, in a process for deriving a collocated block by using a motion vector and a temporal vector of an adjacent block, a motion vector at a lower right position of a block may be determined as a motion vector (ColMv) of the collocated block. Alternatively, when unavailable after checking the lower right position, a motion vector at a center position may be determined as ColMv.

[0190] The M-TMVP proposed in this embodiment can be applied not only to the normal merge mode but also to the AMVP mode, and of course, it can be applied to various inter-frame prediction modes configuring MVP.

[0191] Fig.11 is a diagram illustrating a method for deriving motion information from a reference block according to an embodiment of the present disclosure.

[0192] In the process for deriving SbTMVP, the motion vector (or motion information) of each sub-block present in the collocated block can exist in various types. For example, there may be a unidirectional prediction block or a bidirectional prediction block, or there may be a motion vector in the same direction as the collocated picture, or there may be a motion vector in the opposite direction to the collocated picture.

[0193] refer to Fig.11 , when each sub-block present in the collocated block is a unidirectional prediction block (i.e., only mvColL0 exists) and is a random access (RA) condition, the motion vector of the current block can be derived from the collocated block, such as Fig.11 as shown in .

[0194] When the current picture is RA conditional, the motion vector mvColL0 of the collocated block in the collocated picture can be used to derive the MVP colMvL0 of the current block. Fig.11 As shown in , the final colMvL0 can be derived by using the ratio of the distance between the current picture (currPic) and the reference picture (refPic(L0)) and the distance between the collocated picture (colPic) and the reference picture of the collocated picture (colRefPic(L0)).

[0195] Fig.12 is a diagram illustrating a method for deriving motion information from a reference block according to an embodiment of the present disclosure.

[0196] refer to Fig.12 , when each sub-block present in the collocated block is a unidirectional prediction block (i.e., only mvColL0 exists) and is a low delay (LD) condition, the motion vector of the current block can be derived from the collocated block, such as Fig.12 as shown in .

[0197] When the current picture is in LD condition, the motion vector mvColL0 of the collocated block in the collocated picture can be used to derive colMvLX, X=0,1, such as Fig.12 The final colMvLX can be derived by using the ratio of the distance between the current picture (currPic) and each reference picture (refPic(LX), X is 0 or 1) and the distance between the collocated picture (colPic) and each reference picture of the collocated picture (colRefPic(LX)).

[0198] Generally, in video coding, the purpose of bidirectional prediction is to bring about a smoothing effect by averaging between reference blocks and to improve compression performance by increasing the accuracy of prediction through two motion information. Therefore, in this embodiment, Fig.13 and Fig.14A method for improving compression performance by changing the above process for deriving colMv ​​to generate bidirectional prediction blocks is described in .

[0199] Fig.13 is a diagram illustrating a method of deriving motion information from a reference block according to an embodiment of the present disclosure.

[0200] Can be changed and applied for use in the above Fig.11 The method described in deriving colMv ​​under RA conditions is as follows Fig.13 as shown in .

[0201] refer to Fig.13 , mvColL0 can be used to derive colMvL1 as well as colMvL0. Similarly, colMvL1 can be derived by using directionality and the distance ratio to the reference picture. In this case, colMvL1 is opposite in direction to colMvL0 and can be derived by scaling the absolute value of colMvL0 according to the distance ratio between the reference pictures.

[0202] In addition, as an example, when DefaultMV includes bidirectional motion information, the motion vector can be configured by using the L(1-X) motion vector of DefaultMV for the sub-block performing LX unidirectional prediction. By this, a bidirectional prediction effect can be expected.

[0203] Fig.14 is a diagram illustrating a method for deriving motion information from a reference block according to an embodiment of the present disclosure.

[0204] Can be changed and applied for use in the above Fig.12 The method for deriving colMv ​​under LD conditions described in Fig.14 as shown in .

[0205] refer to Fig.14 ,Under LD conditions, unidirectional prediction blocks can be configured to have only unidirectional motion without being forced to switch to bidirectional prediction mode. This can reduce the computational complexity and maintain consistency with RA.

[0206] Alternatively, as an example, it may be applied as follows without considering the RA / LD condition. For example, when the motion information of the unidirectional prediction block is applied to the current block, the motion vector may be scaled and applied to a reference picture having a close distance between the current picture and the reference picture.

[0207] In addition, according to an embodiment of the present disclosure, information specifying a collocated picture may be signaled based on a higher level syntax. With respect to the prior art, TMVP and SbTMVP determine collocated pictures based on information parsed from a picture header and a slice header, as shown in Tables 2 and 3 below. The signaled information is applied equally to all temporal candidate derivation processes. Table 2 represents the picture header syntax, and Table 3 represents the slice header syntax.

[0208] [Table 2]

[0209]

[0210] [Table 3]

[0211]

[0212] Referring to Table 2 and Table 3, according to the prior art, although TMVP and SbTMVP have different characteristics, TMVP and SbTMVP derive temporal candidates by using only the same collocated pictures, so there is a problem that the different characteristics between TMVP and SbTMVP cannot be effectively reflected.

[0213] Therefore, an embodiment of the present disclosure may signal the juxtaposed image of SbTMVP separately from TMVP so that SbTMVP may have a separate juxtaposed image. As an embodiment, as shown in Tables 4 and 5 below, the juxtaposed image of SbTMVP may signal the juxtaposed image of TMVP separately.

[0214] [Table 4]

[0215]

[0216] [Table 5]

[0217]

[0218] Table 4 represents a picture header syntax and Table 5 represents a slice header syntax. Referring to Table 4, syntax elements ph_sb_collocated_from_l0_flag and ph_sb_collocated_ref_idx showing collocated pictures for SbTMVP may be defined / signaled. Referring to Table 5, syntax elements sh_sb_collocated_from_l0_flag and sh_sb_collocated_ref_idx showing collocated pictures for SbTMVP may be defined / signaled.

[0219] Fig.15 A schematic configuration of the inter-frame predictor 332 performing the inter-frame prediction method according to the present disclosure is shown.

[0220] refer to Fig.15 , the intra predictor 331 may include a temporal vector deriving unit 1500 , a collocated block determining unit 1510 , a motion vector deriving unit 1520 , and a prediction sample generating unit 1530 .

[0221] The temporal vector deriving unit 1500 may derive a temporal vector of the current block based on a motion vector of one candidate in a candidate group including a plurality of candidates.

[0222] Unlike TMVP, which derives motion information of a block at a predefined position (i.e., the lower right position or the center position of the current block) within a predefined reference picture (or collocated picture, colPic) as a motion vector predictor (MVP), SbTMVP in the sub-block merge mode may use motion information of a collocated block (or col block) derived from motion information of a neighboring block (e.g., a left neighboring block of the current block) as the MVP of the current block and derive the MVP in a sub-block unit.

[0223] An improved method in the SbTMVP derivation process considering various time vectors for specifying collocated blocks can be applied. The proposed method can be applied not only to the process for deriving sub-block-based MVP, but also to the process for deriving MVP in the MERGE / AMVP process in substantially the same manner.

[0224] As described above, in order to improve the performance of the existing SbTMVP that uses the motion information of the collocated block derived from the motion information of the left neighboring block as the MVP, the best MVP can be derived by considering various motion information derived from multiple neighboring blocks. In this case, the Figure 6 and Figure 7 In this regard, overlapping descriptions are omitted.

[0225] In addition, as described above, when adding a candidate group to the candidates based on multiple neighboring blocks, an overlap check may be performed to determine whether it has a motion vector that overlaps with a candidate previously included in the candidate group. Figure 8 As described in , whether there is overlap can be determined based on a comparison between the difference between the motion vectors and a threshold.

[0226] The collocated block determination unit 1510 may determine (or specify) a collocated block of a current block in a collocated picture based on the derived time vector. As an example, a collocated picture may be defined equally for an encoding device and a decoding device. Alternatively, as an example, a collocated picture may be signaled through a higher level syntax. The collocated block of the current block may be determined by a time vector within a collocated picture.

[0227] The motion vector deriving unit 1520 may derive the motion vector of the current block in sub-block units based on the motion vector of the collocated block. In other words, the motion vector deriving unit 1520 may derive (or determine) the motion vector of the sub-block in the collocated block specified by the time vector as the motion vector of the corresponding sub-block in the current block.

[0228] The prediction sample generation unit 1530 may perform inter-frame prediction on the current block based on the motion vector of the current block. The prediction sample generation unit 1530 may generate a prediction block of the current block by performing inter-frame prediction based on motion information derived in sub-block units within the current block. As an embodiment, the prediction block of the current block may be generated in sub-block units according to the motion information derived in sub-block units.

[0229] Fig.16 An inter-frame prediction method performed by the encoding apparatus 200 as an embodiment of the present disclosure is shown.

[0230] In an embodiment of the present disclosure, an inter-frame prediction method performed by an encoding device is described. When performing inter-frame prediction by using SbTMVP, the above can be applied in substantially the same manner. Figures 4 to 15 and in this case, repeated description is omitted.

[0231] refer to Fig.16 , the encoding device may determine a time vector of the current block based on a motion vector of one candidate in a candidate group including a plurality of candidates S1601.

[0232] Unlike TMVP, which derives motion information of a block at a predefined position (i.e., a lower right position or a center position of a current block) within a predetermined reference picture (or collocated picture, colPic) as a motion vector predictor (MVP), SbTMVP in sub-block merge mode can use motion information of a collocated block (or col block) derived from motion information of a neighboring block (e.g., a left neighboring block of the current block) as the MVP of the current block and derive the MVP in a sub-block unit.

[0233] An improved method in the SbTMVP derivation process considering various time vectors for specifying collocated blocks can be applied. The proposed method can be applied not only to the process for deriving sub-block-based MVP, but also to the process for deriving MVP in the MERGE / AMVP process in substantially the same manner.

[0234] As described above, in order to improve the performance of the existing SbTMVP that uses the motion information of the collocated block derived from the motion information of the left neighboring block as the MVP, the best MVP can be derived by considering various motion information derived from multiple neighboring blocks. In this case, Figure 6 and Figure 7 In this regard, overlapping descriptions are omitted.

[0235] Additionally, as described above, a template matching method may be applied when deriving the optimal MVP.

[0236] In addition, as described above, before adding a candidate group to the candidates based on multiple neighboring blocks, an overlap check may be performed to determine whether it has a motion vector that overlaps with a candidate previously included in the candidate group. Figure 8 As described in , whether there is overlap can be determined based on a comparison between the difference between the motion vectors and a threshold.

[0237] The encoding device may determine (or specify) a collocated block of the current block in the collocated picture based on the derived time vector S1610. As an example, the collocated picture may be defined identically for the encoding device and the decoding device. Alternatively, as an example, the collocated picture may be signaled through a higher level syntax. The collocated block of the current block may be determined by the time vector within the collocated picture.

[0238] The encoding device may derive a motion vector of the current block in subblock units based on the motion vector of the collocated block S1620. In other words, the encoding device may derive (or determine) a motion vector of a subblock in the collocated block specified by the time vector as a motion vector of a corresponding subblock in the current block.

[0239] The encoding device may perform inter-frame prediction on the current block based on the motion vector of the current block S1630. The encoding device may generate a prediction block of the current block by performing inter-frame prediction based on motion information derived in sub-block units within the current block. As an embodiment, the prediction block of the current block may be generated in sub-block units according to the motion information derived in the sub-block units.

[0240] Fig.17 A schematic configuration of the inter-frame predictor 221 performing the inter-frame prediction method according to the present disclosure is shown.

[0241] refer to Fig.17 , the inter-frame predictor 221 may include a temporal vector determination unit 1700 , a collocated block determination unit 1710 , a motion vector determination unit 1720 , and a prediction sample generation unit 1730 .

[0242] Specifically, the time vector determining unit 1700 may determine the time vector of the current block based on a motion vector of one candidate in a candidate group including a plurality of candidates.

[0243] Unlike TMVP, which derives motion information of a block at a predefined position (i.e., a lower right position or a center position of a current block) within a predetermined reference picture (or collocated picture, colPic) as a motion vector predictor (MVP), SbTMVP in sub-block merge mode may use motion information of a collocated block (or col block) derived from motion information of a neighboring block (e.g., a left neighboring block of the current block) as the MVP of the current block and derive the MVP in a sub-block unit.

[0244] An improved method that considers various time vectors for specifying collocated blocks in the SbTMVP derivation process can be applied. The proposed method can be applied not only to the process for deriving sub-block-based MVP, but also to the process for deriving MVP in the MERGE / AMVP process in substantially the same manner.

[0245] As described above, in order to improve the performance of the existing SbTMVP that uses the motion information of the collocated block derived from the motion information of the left neighboring block as the MVP, the best MVP can be derived by considering various motion information derived from multiple neighboring blocks. In this case, Figure 6 and Figure 7 In this regard, repeated descriptions are omitted.

[0246] Additionally, as described above, a template matching method may be applied when deriving the optimal MVP.

[0247] In addition, as described above, when adding a candidate group to a candidate based on a plurality of neighboring blocks, an overlap check may be performed to determine whether it has a motion vector that overlaps with a candidate previously included in the candidate group. Figure 8 As described in , whether there is overlap can be determined based on a comparison between the difference between the motion vectors and a threshold.

[0248] The collocated block determination unit 1710 may determine (or specify) a collocated block of the current block in the collocated picture based on the derived time vector. As an example, the collocated picture may be defined equally for the encoding device and the decoding device. Alternatively, as an example, the collocated picture may be signaled by a higher level syntax. The collocated block of the current block may be determined by a time vector within the collocated picture.

[0249] The motion vector determination unit 1720 may derive the motion vector of the current block in sub-block units based on the motion vector of the collocated block. In other words, the motion vector determination unit 1720 may derive (or determine) the motion vector of the sub-block in the collocated block specified by the time vector as the motion vector of the corresponding sub-block in the current block.

[0250] The prediction sample generation unit 1730 may perform inter-frame prediction on the current block based on the motion vector of the current block. The prediction sample generation unit 1730 may generate a prediction block of the current block by performing inter-frame prediction based on motion information derived in sub-block units within the current block. As an embodiment, the prediction block of the current block may be generated in sub-block units according to the motion information derived in the sub-block units.

[0251] In the above embodiments, the method is described as a series of steps or boxes based on the flowchart, but the corresponding embodiments are not limited to the order of the steps, and some steps may occur simultaneously or in a different order than other steps described above. In addition, those skilled in the art will appreciate that the steps shown in the flowchart are not exclusive, and other steps may be included or one or more steps in the flowchart may be deleted without affecting the scope of the embodiments of the present disclosure.

[0252] The above-mentioned method according to the embodiment of the present disclosure can be implemented in the form of software, and the encoding device and / or decoding device according to the present disclosure can be included in a device that performs image processing, such as a TV, a computer, a smart phone, a set-top box, a display device, etc.

[0253] In the present disclosure, when the embodiment is implemented as software, the above method can be implemented as a module (process, function, etc.) that performs the above functions. The module can be stored in a memory and can be executed by a processor. The memory can be located inside or outside the processor and can be connected to the processor by various well-known means. The processor may include an application-specific integrated circuit (ASIC), another chipset, a logic circuit, and / or a data processing device. The memory may include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium, and / or other storage devices. In other words, the embodiments described herein can be implemented on a processor, a microprocessor, a controller, or a chip. For example, the functional unit shown in each of the figures can be implemented on a computer, a processor, a microprocessor, a controller, or a chip. In this case, information (e.g., information about instructions) or an algorithm for implementation can be stored in a digital storage medium.

[0254] In addition, the decoding device and the encoding device of the embodiment of the present disclosure can be included in multimedia broadcast sending and receiving devices, mobile communication terminals, home theater video devices, digital theater video devices, surveillance cameras, video conversation devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, devices for providing video on demand (VoD) services, over-the-top video (OTT) devices, devices for providing Internet streaming services, three-dimensional (3D) video devices, virtual reality (VR) devices, augmented reality (AR) devices, videophone video devices, transportation terminals (e.g., vehicle (including autonomous driving vehicle) terminals, aircraft terminals, ship terminals, etc.) and medical video devices, etc., and can be used to process video signals or data signals. For example, over-the-top video (OTT) devices can include game consoles, Blu-ray players, networked TVs, home theater systems, smart phones, tablet computers, digital video recorders (DVRs), etc.

[0255] In addition, the processing method of the embodiment of the present disclosure can be generated in the form of a program executed by a computer and can be stored in a computer-readable recording medium. Multimedia data having a data structure according to an embodiment of the present disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices and distributed storage devices that store computer-readable data. Computer-readable recording media may include, for example, Blu-ray discs (BD), universal serial buses (USB), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, tapes, floppy disks, and optical media storage devices. In addition, computer-readable recording media include media implemented in the form of carrier waves (e.g., transmitted via the Internet). In addition, the bit stream generated by the encoding method may be stored in a computer-readable recording medium or may be sent via a wired or wireless communication network.

[0256] In addition, the embodiments of the present disclosure may be implemented by a computer program product through a program code, and the program code may be executed on a computer by the embodiments of the present disclosure. The program code may be stored on a computer-readable carrier.

[0257] Fig.18 An example of a content streaming system to which an embodiment of the present disclosure can be applied is shown.

[0258] refer to Fig.18 The content streaming transmission system to which the embodiments of the present disclosure are applied may mainly include an encoding server, a streaming transmission server, a web server, a media storage, a user device, and a multimedia input device.

[0259] The encoding server generates a bitstream by compressing content input from a multimedia input device such as a smartphone, a camera, a camcorder, etc. into digital data and transmits it to the streaming server. As another example, when a multimedia input device such as a smartphone, a camera, a camcorder, etc. directly generates a bitstream, the encoding server may be omitted.

[0260] A bitstream may be generated by applying the encoding method or the bitstream generating method of the embodiment of the present disclosure, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0261] The streaming server sends multimedia data to the user device through the web server based on the user's request, and the web server serves as a medium to inform the user what services are available. When the user requests the required service from the web server, the web server delivers it to the streaming server, and the streaming server sends the multimedia data to the user. In this case, the content streaming system may include a separate control server, and in this case, the control server controls the command / response between each device in the content streaming system.

[0262] The streaming server may receive content from a media storage and / or encoding server. For example, when receiving content from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bitstream for a specific period of time.

[0263] Examples of user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation, tablet PCs, tablet computers, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays (HMDs), digital televisions, desktops, digital signage, etc.).

[0264] Each server in the content streaming system may be operated as a distributed server, and in this case, data received from each server may be distributed and processed.

[0265] The claims set forth herein can be combined in various ways. For example, the technical features of the method claims of the present disclosure can be combined and implemented as a device, and the technical features of the device claims of the present disclosure can be combined and implemented as a method. In addition, the technical features of the method claims of the present disclosure and the technical features of the device claims can be combined and implemented as a device, and the technical features of the method claims of the present disclosure and the technical features of the device claims can be combined and implemented as a method.

Claims

1. An image decoding method, the method comprising: deriving a temporal vector of the current block based on a motion vector of a candidate in a candidate group including a plurality of candidates; Based on the time vector, determining a collocated block of the current block in a collocated picture; deriving a motion vector of the current block in a sub-block unit based on the motion vector of the collocated block; as well as Based on the motion vector of the current block, inter prediction for the current block is performed.

2. The method according to claim 1, wherein: The candidate group includes adjacent spatial neighboring blocks of the current block and non-adjacent spatial neighboring blocks of the current block as candidates.

3. The method according to claim 2, wherein: The non-adjacent spatially neighboring blocks include: a first block including samples separated from an upper left sample adjacent to an upper left corner of the current block in a left sample line adjacent to the current block by a value obtained by multiplying a height of the current block by 2; and a second block including samples separated from the upper left sample in a top sample line adjacent to the current block by a value obtained by multiplying a width of the current block by 2.

4. The method according to claim 1, wherein: The one candidate is selected from among the plurality of candidates included in the candidate group based on template matching.

5. The method according to claim 4, wherein: The template matching cost is calculated based on a sum of absolute differences (SAD) or a mean sum of absolute differences (MRSAD) between a template region of the current block and template regions of blocks specified by the plurality of candidate motion vectors included in the candidate group.

6. The method according to claim 4, wherein: The template matching cost is calculated based on a sum of absolute differences (SAD) or a mean sum of absolute differences (MRSAD) between a template area of ​​the current block and a template area of ​​a block specified by a motion vector of a subblock or block derived based on the plurality of candidates included in the candidate group.

7. The method according to claim 1, further comprising: configuring the candidate group including the plurality of candidates based on neighboring blocks of the current block, The candidate group is configured by adding neighboring blocks at specific positions to the candidate group in a predefined order.

8. The method according to claim 7, wherein: Configuring the candidate group includes confirming whether the motion vector of the neighboring block at the specific position overlaps with a motion vector of a candidate previously included in the candidate group.

9. The method according to claim 8, wherein: Whether the overlap exists is determined based on whether a difference between a motion vector of the candidate previously included in the candidate group and a motion vector of the neighboring block at the specific position is less than a predefined threshold.

10. The method of claim 1, wherein: determining the collocated picture based on a syntax element signaled by at least one syntax of a picture header or a slice header, The syntax element is signaled separately from the syntax element indicating the collocated picture for the temporal motion vector predictor.

11. A method for image encoding, the method comprising: determining a temporal vector of the current block based on a motion vector of a candidate in a candidate group including a plurality of candidates; determining a collocated block of the current block in a collocated picture based on the time vector; determining a motion vector of the current block in a sub-block unit based on a motion vector of the collocated block; as well as Inter prediction for the current block is performed based on the motion vector of the current block.

12. A computer-readable storage medium storing a bit stream generated by the image encoding method according to claim 11.

13. A method for transmitting data for image information, the method comprising: determining a temporal vector of the current block based on a motion vector of a candidate in a candidate group including a plurality of candidates; determining a collocated block of the current block in a collocated picture based on the time vector; determining a motion vector of the current block in a sub-block unit based on a motion vector of the collocated block; generating a prediction sample of the current block by performing inter-frame prediction for the current block based on a motion vector of the current block; generating a bitstream by encoding the current block based on the prediction block; as well as Data including the bit stream is transmitted.

Images