Image encoding / decoding method and device, and recording medium storing bit stream

Abstract

An image decoding / encoding method and apparatus according to the present disclosure may generate a candidate list for a current block, reorder one or more candidates included in the candidate list based on a specific cost, and derive motion information on the current block based on the candidate list such that inter prediction is performed. In this aspect, a cost may be calculated for each of a plurality of candidates included in the candidate list according to at least one of a template matching-based calculation method, a bidirectional matching-based calculation method, or a motion information-based calculation method.

Description

Technical Field

[0001] The present disclosure relates to an image encoding / decoding method and apparatus and a recording medium storing a bit stream. Background Art

[0002] Recently, demands for high-resolution and high-quality images such as HD (High Definition) images and UHD (Ultra High Definition) images have been increasing in various application fields, and therefore, efficient image compression technology is being discussed.

[0003] There are various technologies, such as inter-frame prediction technology that uses video compression technology to predict pixel values ​​included in the current picture from pictures before or after the current picture, intra-frame prediction technology that predicts pixel values ​​included in the current picture by using pixel information in the current picture, entropy coding technology that assigns short symbols to values ​​with high occurrence frequency and assigns long symbols to values ​​with low occurrence frequency, etc., and these image compression technologies can be used to effectively compress image data and transmit or store it. Summary of the invention

[0004] Technical issues

[0005] The present disclosure provides a method and apparatus for reordering candidates in a candidate list for inter-frame prediction.

[0006] The present disclosure provides a method and apparatus for calculating a cost for reordering a candidate list.

[0007] The present disclosure provides a plurality of cost calculation methods for reordering a candidate list, and further provides a method and apparatus for selectively using any one of the plurality of cost calculation methods.

[0008] Technical Solution

[0009] According to the image decoding method and device of the present disclosure, a candidate list for the current block can be generated, one or more candidates in the candidate list can be reordered based on a predetermined cost, motion information of the current block can be derived based on the candidate list and the candidate index, and inter-frame prediction can be performed on the current block based on the motion information.

[0010] In the image decoding method and apparatus according to the present disclosure, a cost is calculated for each of a plurality of candidates in a candidate list according to at least one of a calculation method based on template matching or a calculation method based on bilateral matching.

[0011] In the image decoding method and device according to the present disclosure, one of the calculation method based on template matching or the calculation method based on bilateral matching can be selectively used based on whether the current block meets the block size condition.

[0012] In the image decoding method and apparatus according to the present disclosure, one of a calculation method based on template matching or a calculation method based on bilateral matching may be selectively used based on whether a candidate belonging to a candidate list satisfies a bidirectional prediction condition.

[0013] In the image decoding method and device according to the present disclosure, the bidirectional prediction condition may include at least one of a first condition, a second condition, a third condition, a fourth condition or a fifth condition, the first condition being that the candidate motion information is motion information for bidirectional prediction, the second condition being that one of the L0 reference picture or the L1 reference picture is temporally before the current picture and the other is temporally after the current picture, the third condition being that the temporal distance between the current picture and the L0 reference picture is the same as the temporal distance between the current picture and the L1 reference picture, the fourth condition being that the candidate L0 and L1 reference pictures use weighted prediction, and the fifth condition being that the candidate L0 and L1 reference pictures have the same size as the current picture.

[0014] In the image decoding method and apparatus according to the present disclosure, one of a calculation method based on template matching or a calculation method based on bilateral matching may be selectively used based on whether the current block is a block coded in a sub-block mode.

[0015] In the image decoding method and apparatus according to the present disclosure, one of a calculation method based on template matching or a calculation method based on bilateral matching may be selectively used based on position information of the current block.

[0016] In the image decoding method and device according to the present disclosure, one of the calculation methods based on template matching or the calculation methods based on bilateral matching can be selectively used based on whether the current block satisfies the block size condition, whether the candidates in the candidate list satisfy the bidirectional prediction condition, whether the current block is a block coded in the sub-block mode, or at least two of the position information of the current block.

[0017] In the image decoding method and apparatus according to the present disclosure, when the current block satisfies the block size condition and the candidates in the candidate list satisfy the bidirectional prediction condition, a calculation method based on bilateral matching may be used to calculate the cost.

[0018] In the image decoding method and apparatus according to the present disclosure, when the current block is coded in the sub-block mode and the candidate satisfies the bidirectional prediction condition, the cost may be calculated using a calculation method based on bilateral matching.

[0019] In the image decoding method and apparatus according to the present disclosure, when the current block exists at a position of a template area not configured for a calculation method based on template matching and the candidate satisfies a bidirectional prediction condition, a calculation method based on bilateral matching may be used to calculate the cost.

[0020] In the image decoding method and device according to the present disclosure, the calculation method based on bilateral matching can use the L0 reference block and L1 reference block specified based on the motion information of the candidate belonging to the candidate list to calculate the cost. Here, the cost can be calculated based on some pixels in the L0 reference block and the L1 reference block.

[0021] In the image decoding method and device according to the present disclosure, some pixels may be determined as subsampled pixel columns in the L0 reference block and the L1 reference block, subsampled subblocks in the L0 reference block and the L1 reference block, or subsampled pixel columns in a subsampled subblock.

[0022] According to the image encoding method and apparatus of the present disclosure, a candidate list of a current block can be generated, one or more candidates in the candidate list can be reordered based on a predetermined cost, and inter-frame prediction can be performed on the current block based on the candidate list. Here, the cost can be calculated for each of the multiple candidates in the candidate list according to at least one of a calculation method based on template matching or a calculation method based on bilateral matching.

[0023] A computer-readable digital storage medium is provided, which stores encoded video / image information, thereby causing a decoding device according to the present disclosure to perform an image decoding method.

[0024] A computer-readable digital storage medium storing video / image information generated according to an image encoding method according to the present disclosure is provided.

[0025] A method and apparatus for transmitting video / image information generated according to an image encoding method according to the present disclosure are provided.

[0026] Beneficial Effects

[0027] According to the present disclosure, by reordering one or more candidates belonging to a candidate list, the encoding efficiency of the candidate index can be improved.

[0028] According to the present disclosure, by adaptively using a calculation method based on bilateral matching or a calculation method based on motion information to calculate a cost, it is possible to reduce a memory bandwidth required for cost calculation.

[0029] According to the present disclosure, by selectively using only some pixels of a target block used to calculate a cost for a re-ordered candidate list, the complexity of cost calculation can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A video / image compilation system according to the present disclosure is shown.

[0031] Figure 2A schematic block diagram showing an encoding device to which an embodiment of the present disclosure is applicable and which performs encoding of a video / image signal.

[0032] Figure 3 A schematic block diagram showing a decoding device to which an embodiment of the present disclosure is applicable and which performs decoding of a video / image signal.

[0033] Figure 4 An inter-frame prediction method performed by a decoding device according to an embodiment of the present disclosure is shown.

[0034] Figure 5 A schematic configuration of the inter-frame predictor 332 that performs the inter-frame prediction method according to the present disclosure is shown.

[0035] Figure 6 An inter-frame prediction method performed by an encoding device as an embodiment according to the present disclosure is shown.

[0036] Figure 7 A schematic configuration of the inter-frame predictor 221 that performs the inter-frame prediction method according to the present disclosure is shown.

[0037] Figure 8 An example of a content streaming system to which an embodiment of the present disclosure can be applied is shown. DETAILED DESCRIPTION

[0038] Because the present disclosure can make various changes and has several embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, it is not intended to limit the present disclosure to specific embodiments, and it should be understood to include all changes, equivalents and substitutes included in the spirit and technical scope of the present disclosure. When describing each of the drawings, similar reference numerals are used for similar components.

[0039] Terms such as first, second, etc. can be used to describe various components, but components should not be limited by these terms. These terms are only used to distinguish one component from other components. For example, without departing from the scope of the present disclosure, a first component can be referred to as a second component, and similarly, a second component can also be referred to as a first component. Terms and / or combinations of any one or more related statement items in a plurality of related statement items are included.

[0040] When a component is referred to as being "connected" or "linked" to another component, it should be understood that it can be directly connected or linked to another component, but another component may also exist in between. On the other hand, when a component is referred to as being "directly connected" or "directly linked" to another component, it should be understood that another component does not exist in between.

[0041] The terms used in this application are only used to describe specific embodiments and are not intended to limit the present disclosure. Unless the context clearly indicates otherwise, singular expressions include plural expressions. In this application, it should be understood that terms such as "including" or "having" are intended to designate the existence of features, numbers, steps, operations, components, parts or combinations thereof described in the specification, but do not exclude the possibility of the existence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof in advance.

[0042] The present disclosure relates to video / image coding. For example, the methods / embodiments disclosed herein may be applied to methods disclosed in the Universal Video Coding (VVC) standard. In addition, the methods / embodiments disclosed herein may be applied to methods disclosed in the Basic Video Coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the second generation audio video coding standard (AVS2), or the next generation video / image coding standard (e.g., H.267 or H.268, etc.).

[0043] This specification proposes various embodiments of video / image coding, and unless otherwise stated, these embodiments may be performed in combination with each other.

[0044] Here, video may refer to a collection of a series of images over time. A picture generally refers to a unit representing an image within a specific time period, and a slice / tile is a unit that forms a part of a picture in coding. A slice / tile may include at least one coding tree unit (CTU). A picture may be composed of at least one slice / tile. A tile is a rectangular area consisting of multiple CTUs within a specific tile column and a specific tile row of a picture. A tile column is a rectangular area of ​​a CTU having the same height as the picture and a width assigned by the syntax requirements of the picture parameter set. A tile row is a rectangular area of ​​a CTU having a height assigned by the picture parameter set and the same width as the width of the picture. The CTU within a tile may be arranged continuously according to a CTU raster scan, and the tiles within a picture may be arranged continuously according to a raster scan of the tile. A slice may include an integer number of complete tiles or an integer number of continuous complete CTU rows within a tile of a picture that may be exclusively included in a single NAL unit. At the same time, a picture may be divided into at least two sub-pictures. A sub-picture may be a rectangular area of ​​at least one slice within a picture.

[0045] Pixel, pixel or picture element may refer to the smallest unit constituting a picture (or image). In addition, "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luminance component, or only a pixel / pixel value of a chrominance component.

[0046] A unit may represent a basic unit of image processing. A unit may include at least one of a specific region of a picture and information associated with the corresponding region. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, a unit may be used interchangeably with terms such as a block or region. In general, an MxN block may include a set (or array) of transform coefficients or samples (or sample arrays) consisting of M columns and N rows.

[0047] Here, "A or B" may refer to "only A", "only B", or "both A and B". In other words, herein, "A or B" may be interpreted as "A and / or B". For example, herein, "A, B or C" may refer to "only A", "only B", "only C", or "any combination of A, B, and C".

[0048] As used herein, a slash ( / ) or a comma may mean "and / or". For example, "A / B" may mean "A and / or B". Thus, "A / B" may mean "only A", "only B", or "both A and B". For example, "A, B, C" may mean "A, B, or C".

[0049] Here, "at least one of A and B" may refer to "only A", "only B", or "both A and B". In addition, herein, expressions such as "at least one of A or B" or "at least one of A and / or B" can be interpreted in the same manner as "at least one of A and B".

[0050] In addition, herein, “at least one of A, B, and C” may refer to “only A”, “only B”, “only C”, or “any combination of A, B, and C”. In addition, “at least one of A, B, or C” or “at least one of A, B and / or C” may refer to “at least one of A, B, and C”.

[0051] In addition, the brackets used herein may refer to "for example". Specifically, when the indication is "prediction (intra-frame prediction)", "intra-frame prediction" may be proposed as an example of "prediction". In other words, the "prediction" here is not limited to "intra-frame prediction", and "intra-frame prediction" may be proposed as an example of "prediction". In addition, even when the indication is "prediction (ie, intra-frame prediction)", "intra-frame prediction" may be proposed as an example of "prediction".

[0052] Here, technical features described individually in one drawing may be implemented individually or simultaneously.

[0053] Figure 1 A video / image compilation system according to the present disclosure is shown.

[0054] refer to Figure 1, a video / image coding system may include a first device (source device) and a second device (receiving device).

[0055] The source device may send the encoded video / image information or data to the receiving device in the form of a file or stream transmission through a digital storage medium or a network. The source device may include a video source, an encoding device, and a sending unit. The receiving device may include a receiving unit, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be composed of a separate device or an external component.

[0056] The video source may obtain the video / image through the process of capturing, synthesizing or generating the video / image. The video source may include a device for capturing the video / image and a device for generating the video / image. The device for capturing the video / image may include at least one camera, a video / image archive including previously captured videos / images, etc. The device for generating the video / image may include a computer, a tablet computer, a smart phone, etc., and may (electronically) generate the video / image. For example, a virtual video / image may be generated by a computer, etc., and in this case, the process of capturing the video / image may be replaced by the process of generating the relevant data.

[0057] The encoding device can encode the input video / image. The encoding device can perform a series of processes such as prediction, transformation, quantization, etc. for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bit stream.

[0058] The sending unit may send the encoded video / image information or data output in the form of a bit stream to the receiving unit of the receiving device in the form of a file or stream transmission through a digital storage medium or a network. The digital storage medium may include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The sending unit may include an element for generating a media file in a predetermined file format and may include an element for transmission through a broadcast / communication network. The receiving unit may receive / extract a bit stream and send it to a decoding device.

[0059] The decoding device can decode the video / image by performing a series of processes such as inverse quantization, inverse transformation, prediction, etc. corresponding to the operation of the encoding device.

[0060] The renderer may render the decoded video / image. The rendered video / image may be displayed through a display unit.

[0061] Figure 2A rough block diagram of an encoding device to which an embodiment of the present disclosure can be applied and which performs encoding of a video / image signal is shown.

[0062] refer to Figure 2 , the encoding device 200 may be composed of an image segmenter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, an inverse quantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. According to an embodiment, the above-mentioned image segmenter 210, the predictor 220, the residual processor 230, the entropy encoder 240, the adder 250, and the filter 260 may be configured by at least one hardware component (e.g., an encoder chipset or processor). In addition, the memory 270 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include a memory 270 as an internal / external component.

[0063] The image divider 210 may divide the input image (or picture, frame) input to the encoding device 200 into at least one processing unit. As an example, the processing unit may be referred to as a coding unit (CU). In this case, the coding unit may be recursively divided from a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree binary tree ternary tree (QTBTTT) structure.

[0064] For example, one coding unit may be segmented into a plurality of coding units having a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary structure. In this case, for example, the quadtree structure may be applied first, and the binary tree structure and / or the ternary structure may be applied later. Alternatively, the binary tree structure may be applied before the quadtree structure. The coding process according to this specification may be performed based on a final coding unit that is no longer segmented. In this case, based on coding efficiency according to image characteristics, etc., the maximum coding unit may be directly used as the final coding unit, or if necessary, the coding unit may be recursively segmented into coding units of a deeper depth, and the coding unit with the best size may be used as the final coding unit. Here, the coding process may include processes such as prediction, transformation, and reconstruction described later.

[0065] As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be divided or partitioned from the above-mentioned final coding unit, respectively. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient.

[0066] In some cases, a unit may be used interchangeably with terms such as a block or region. In general, an MxN block may represent a set of transform coefficients or samples consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luma component, or only a pixel / pixel value of a chroma component. A sample may be used as a term to make a picture (or image) correspond to a pixel or a picture element.

[0067] The encoding device 200 may subtract the prediction signal (prediction block, prediction sample array) output from the inter predictor 221 or the intra predictor 222 from the input image signal (original block, original sample array) to generate a residual signal (residual signal, residual sample array), and the generated residual signal is sent to the transformer 232. In this case, the unit that subtracts the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) within the encoding device 200 may be referred to as a subtractor 231.

[0068] The predictor 220 may perform prediction on a block to be processed (hereinafter referred to as a current block) and generate a predicted block including a prediction sample for the current block. The predictor 220 may determine whether intra prediction or inter prediction is applied in units of a current block or CU. The predictor 220 may generate various information about the prediction, such as prediction mode information, and send it to the entropy encoder 240, as described later in the description of each prediction mode. The information about the prediction may be encoded in the entropy encoder 240 and output in the form of a bitstream.

[0069] The intra-frame predictor 222 can predict the current block by referring to the samples in the current picture. Depending on the prediction mode, the referenced sample can be located near the current block or can be located at a specific distance away from the current block. In intra-frame prediction, the prediction mode may include at least one non-directional mode and multiple directional modes. The non-directional mode may include at least one of the DC mode or the plane mode. Depending on the detail level of the prediction direction, the directional mode may include 33 directional modes or 65 directional modes. However, this is only an example, and more or less directional modes may be used depending on the configuration. The intra-frame predictor 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0070] The inter-frame predictor 221 may derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter-frame predictor 221 may configure a motion information candidate list based on the neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, and for example, for skip mode and merge mode, the inter predictor 221 may use motion information of a neighboring block as motion information of a current block. For skip mode, unlike merge mode, a residual signal may not be transmitted. For motion vector prediction (MVP) mode, a motion vector of a neighboring block is used as a motion vector predictor, and a motion vector difference is signaled to indicate a motion vector of the current block.

[0071] The predictor 220 may generate a prediction signal based on various prediction methods described later. For example, the predictor may not only apply intra prediction or inter prediction to predict a block, but may also apply intra prediction and inter prediction at the same time. It may be referred to as a combined inter and intra prediction (CIIP) mode. In addition, the predictor may be based on an intra block copy (IBC) prediction mode or may be based on a palette mode for prediction of a block. The IBC prediction mode or the palette mode may be used for content image / video coding of games, such as screen content coding (SCC), etc. IBC basically performs prediction within the current picture, but it may be performed similarly to inter prediction because it derives a reference block within the current picture. In other words, IBC may use at least one of the inter prediction techniques described herein. The palette mode may be considered an example of intra coding or intra prediction. When the palette mode is applied, the sample values ​​within the picture may be signaled based on information about the palette table and the palette index. The prediction signal generated by the predictor 220 may be used to generate a reconstructed signal or a residual signal.

[0072] The transformer 232 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loève transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, GBT refers to a transform obtained from a graph when relationship information between pixels is expressed as a graph. CNT refers to a transform obtained based on generating a prediction signal using all previously reconstructed pixels. In addition, the transform process may be applied to square pixel blocks of the same size or may be applied to non-square blocks of variable size.

[0073] The quantizer 233 may quantize the transform coefficients and send them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantizer 233 may rearrange the quantized transform coefficients in the block form into a one-dimensional vector form based on the coefficient scanning order, and may generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0074] The entropy encoder 240 may perform various encoding methods such as Exponential Golomb, Context Adaptive Variable Length Coding (CAVLC), Context Adaptive Binary Arithmetic Coding (CABAC), etc. The entropy encoder 240 may encode information necessary for video / video image reconstruction (e.g., values ​​of syntax elements, etc.) in addition to transform coefficients quantized together or individually.

[0075] The encoded information (e.g., encoded video / image information) can be transmitted or stored in units of network abstraction layer (NAL) units in the form of a bitstream. The video / image information may further include information about various parameter sets such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. Here, information and / or syntax elements transmitted / signaled from the encoding device to the decoding device may be included in the video / image information. The video / image information may be encoded and included in the bitstream through the above-mentioned encoding process. The bitstream may be transmitted through a network or may be stored in a digital storage medium. Here, the network may include a broadcast network and / or a communication network, etc., and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmission unit (not shown) for transmission and / or a storage unit (not shown) for storing a signal output from the entropy encoder 240 may be configured as an internal / external element of the encoding device 200, or the transmission unit may also be included in the entropy encoder 240.

[0076] The quantized transform coefficients output from the quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients through the inverse quantizer 234 and the inverse transformer 235. The adder 250 can add the reconstructed residual signal to the prediction signal output from the inter-frame predictor 221 or the intra-frame predictor 222 to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). When there is no residual of the block to be processed, such as when the skip mode is applied, the prediction block can be used as a reconstructed block. The adder 250 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture, and can also be used for inter-frame prediction of the next picture through filtering described later. At the same time, luminance mapping with chroma scaling (LMCS) can be applied in the picture encoding and / or reconstruction process.

[0077] The filter 260 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 260 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and the modified reconstructed picture can be stored in the memory 270, specifically in the DPB of the memory 270. Various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filter 260 can generate various information about filtering and send it to the entropy encoder 240. The information about filtering can be encoded in the entropy encoder 240 and output in the form of a bit stream.

[0078] The modified reconstructed picture transmitted to the memory 270 may be used as a reference picture in the inter predictor 221. When inter prediction is applied therethrough, the encoding apparatus can avoid prediction mismatch in the encoding apparatus 200 and the decoding apparatus, and can also improve encoding efficiency.

[0079] The DPB of the memory 270 may store the modified reconstructed picture to be used as a reference picture in the inter-frame predictor 221. The memory 270 may store the motion information of the block from which the motion information in the current picture is derived (or encoded) and / or the motion information of the block in the pre-reconstructed picture. The stored motion information may be sent to the inter-frame predictor 221 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 270 may store the reconstructed samples of the reconstructed block in the current picture and send them to the intra-frame predictor 222.

[0080] Figure 3 A rough block diagram of a decoding device to which an embodiment of the present disclosure can be applied and which performs decoding of a video / image signal is shown.

[0081] refer to Figure 3 , the decoding apparatus 300 may be configured by including an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 332 and an intra-frame predictor 331. The residual processor 320 may include a dequantizer 321 and an inverse transformer 321.

[0082] According to an embodiment, the above-mentioned entropy decoder 310, residual processor 320, predictor 330, adder 340 and filter 350 may be configured by one hardware component (e.g., decoder chipset or processor). In addition, the memory 360 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory 360 as an internal / external component.

[0083] When a bit stream including video / image information is input, the decoding apparatus 300 may respond to the Figure 2 The image is reconstructed by the process of processing video / image information in the encoding device of the decoding device. For example, the decoding device 300 can derive the unit / block based on the relevant information of the block segmentation obtained from the bit stream. The decoding device 300 can perform decoding by using the processing unit applied in the encoding device. Therefore, the processing unit of decoding can be a coding unit, and the coding unit can be divided from the coding tree unit or the maximum coding unit according to the quadtree structure, the binary tree structure and / or the ternary tree structure. At least one transform unit can be derived from the coding unit. And, the reconstructed image signal decoded and output by the decoding device 300 can be played by a playback device.

[0084] The decoding device 300 may receive the bit stream from Figure 2 The received signal can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) necessary for image reconstruction (or picture reconstruction). The video / image information may further include information about various parameter sets such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. The decoding device may further decode the picture based on the information about the parameter set and / or the general constraint information. The signaled / received information and / or the syntax elements described later in this document may be decoded and obtained from the bitstream by a decoding process. For example, the entropy decoder 310 may decode the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, CABAC, etc., and output the values ​​of the syntax elements necessary for image reconstruction and the quantized values ​​of the transform coefficients of the residual. In more detail, the CABAC entropy decoding method can receive a bin corresponding to each syntax element from a bitstream, determine a context model by using information of syntax elements to be decoded, decoded information of neighboring blocks and blocks to be decoded, or information of symbols / bins decoded in the previous step, perform arithmetic decoding on the bins by predicting the probability of occurrence of the bins according to the determined context model, and generate symbols corresponding to the values ​​of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method can update the context model by using information about the decoded symbols / bins of the context model for the next symbol / bin. Among the information decoded in the entropy decoder 310, information about prediction is provided to the predictor (inter-frame predictor 332 and intra-frame predictor 331), and the residual value, that is, the quantized transform coefficient and related parameter information, which is entropy decoded in the entropy decoder 310, can be input to the residual processor 320. The residual processor 320 can derive a residual signal (residual block, residual sample, residual sample array). In addition, information about filtering among the information decoded in the entropy decoder 310 can be provided to the filter 350. Meanwhile, a receiving unit (not shown) that receives a signal output from the encoding device may be further configured as an internal / external element of the decoding device 300 or the receiving unit may be a component of the entropy decoder 310 .

[0085] Meanwhile, the decoding device according to this specification may be referred to as a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of an inverse quantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.

[0086] The inverse quantizer 321 may inverse quantize the quantized transform coefficient and output the transform coefficient. The inverse quantizer 321 may rearrange the quantized transform coefficient into a two-dimensional block form. In this case, the rearrangement may be performed based on the coefficient scanning order performed in the encoding device. The inverse quantizer 321 may perform inverse quantization on the quantized transform coefficient by using a quantization parameter (e.g., quantization step size information) and obtain the transform coefficient.

[0087] The inverse transformer 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0088] The predictor 320 may perform prediction on the current block and generate a prediction block including prediction samples for the current block. The predictor 320 may determine whether to apply intra prediction or inter prediction to the current block based on the information on prediction output from the entropy decoder 310, and determine a specific intra / inter prediction mode.

[0089] The predictor 320 can generate a prediction signal based on various prediction methods described later. For example, the predictor 320 can not only apply intra prediction or inter prediction to predict a block, but also apply intra prediction and inter prediction at the same time. It can be called a combined inter and intra prediction (CIIP) mode. In addition, the predictor can be based on an intra block copy (IBC) prediction mode or can be based on a palette mode for prediction of a block. The IBC prediction mode or the palette mode can be used for content image / video coding of games, such as screen content coding (SCC), etc. IBC basically performs prediction within the current picture, but it can be performed similarly to inter prediction because it derives a reference block within the current picture. In other words, IBC can use at least one of the inter prediction techniques described herein. The palette mode can be considered as an example of intra coding or intra prediction. When the palette mode is applied, information about the palette table and the palette index can be included in the video / image information and sent with a signal.

[0090] The intra-frame predictor 331 can predict the current block by referring to samples within the current picture. Depending on the prediction mode, the referenced sample can be located near the current block or can be located at a specific distance away from the current block. In intra-frame prediction, the prediction mode can include at least one non-directional mode and multiple directional modes. The intra-frame predictor 331 can determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

[0091] The inter-frame predictor 332 may derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter-frame predictor 332 may configure a motion information candidate list based on the neighboring blocks, and derive a motion vector and / or a reference picture index for the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes, and information about the prediction may include information indicating an inter-frame prediction mode for the current block.

[0092] The adder 340 may add the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including the inter-frame predictor 332 and / or the intra-frame predictor 331) to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). When there is no residual of the block to be processed, such as when the skip mode is applied, the prediction block may be used as the reconstructed block.

[0093] The adder 340 may be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next block to be processed in the current picture, may be output through filtering described later, or may be used for inter prediction of the next picture. At the same time, luminance mapping with chroma scaling (LMCS) may be applied during picture decoding.

[0094] The filter 350 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 350 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and send the modified reconstructed picture to the memory 360, specifically the DPB of the memory 360. The various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0095] The (modified) reconstructed picture stored in the DPB of the memory 360 can be used as a reference picture in the inter-frame predictor 332. The memory 360 can be derived from the motion information in its current picture (or decoded) The motion information of the block and / or the motion information of the block in the pre-reconstructed picture. The stored motion information can be sent to the inter-frame predictor 260 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 360 can store the reconstructed samples of the reconstructed block in the current picture and send them to the intra-frame predictor 331.

[0096] Here, the embodiments described in the filter 260, the inter-frame predictor 221, and the intra-frame predictor 222 of the encoding device 200 may also be equally or correspondingly applied to the filter 350, the inter-frame predictor 332, and the intra-frame predictor 331 of the decoding device 300, respectively.

[0097] Figure 4 An inter-frame prediction method performed by a decoding device according to an embodiment of the present disclosure is shown.

[0098] refer to Figure 4 , a candidate list for predicting / deriving motion information of the current block may be generated ( S400 ).

[0099] The motion information of the current block may include at least one of a motion vector, a reference picture index, inter prediction direction information, or weight information for bidirectional weighted prediction.

[0100] The candidate list for predicting / deriving the motion information of the current block may be used in a merge mode or an advanced motion vector prediction mode (AMVP mode). However, it is not limited thereto, and the candidate list may also be used in an affine merge mode or an affine inter mode. The candidate list for the merge mode of motion merging may be referred to as a merge candidate list. The candidate list for the AMVP mode of motion prediction may be referred to as a prediction candidate list.

[0101] The candidate list may include multiple candidates, and the multiple candidates may include at least one of a spatial candidate or a temporal candidate. Here, the motion information of the spatial candidate may be derived based on the motion information of a neighboring block (hereinafter referred to as a spatial neighboring block) that is spatially adjacent to the current block. The spatial neighboring block may include at least one of an upper neighboring block, a left neighboring block, a lower left neighboring block, an upper right neighboring block, or an upper left neighboring block of the current block. The motion information of the temporal candidate may be derived based on the motion information of a neighboring block (hereinafter referred to as a temporal neighboring block) that is temporally adjacent to the current block. The temporal neighboring block may be a block that belongs to a reference picture (a collocated picture) decoded before the current picture and is a collocated block of the current block. The collocated block may be a block that includes at least one of the position of the upper left sample of the current block, the position of the center sample, or the position of a sample adjacent to the lower right corner of the current block.

[0102] refer to Figure 4 , one or more candidates in the candidate list may be reordered based on a predetermined cost ( S410 ).

[0103] Reordering according to the present disclosure may refer to changing the position of a candidate in a candidate list or changing a candidate index assigned to a candidate. To this end, a cost for reordering may be calculated for each candidate in the candidate list. The candidates may be reordered in ascending or descending order of the calculated cost to generate a final candidate list.

[0104] The cost according to the present disclosure may be calculated according to at least one of a calculation method based on template matching (TM), a calculation method based on bilateral matching (BM), or a calculation method based on motion information, which will be described in detail below.

[0105] Computational method based on template matching

[0106] The cost based on template matching may be calculated as the cost between the template region of the current block and the template region of the reference block. Here, the cost may refer to SAD (Sum of Absolute Difference), SATD (Sum of Absolute Transform Difference), or SSE (Sum of Squared Error).

[0107] The template area of ​​the current block may include at least one of a left neighboring area or an upper neighboring area adjacent to the current block. For example, the template area of ​​the current block may include a left neighboring area adjacent to the current block, and may not include an upper neighboring area. Alternatively, the template area of ​​the current block may include an upper neighboring area adjacent to the current block, and may not include a left neighboring area. Alternatively, the template area of ​​the current block may include an upper neighboring area and a left neighboring area adjacent to the current block.

[0108] The template area of ​​the reference block may be configured to correspond to the template area of ​​the current block. That is, the template area of ​​the reference block may include at least one of a left neighboring area or an upper neighboring area adjacent to the reference block. Here, the reference block may be specified based on the motion information of a candidate belonging to a candidate list of the current block. For example, within a candidate reference picture, a block at a position moved from the position of the current block by a candidate motion vector may be determined as a reference block. Alternatively, within a candidate reference picture, a block at a position moved from a position of a spatial / temporal neighboring block corresponding to the candidate by a candidate motion vector may be determined as a reference block.

[0109] When the candidate motion information is motion information for bidirectional prediction, a reference block of reference picture list 0 (hereinafter referred to as L0 reference block) and a reference block of reference picture list 1 (hereinafter referred to as L1 reference block) may be designated respectively. In this case, a cost between a template area of ​​at least one of the L0 reference block or the L1 reference block and a template area of ​​the current block may be calculated.

[0110] For example, a first cost between a template region of an L0 reference block and a template region of a current block may be calculated, and a second cost between a template region of an L1 reference block and a template region of a current block may be calculated. A final cost may be calculated based on a weighted average of the first cost and the second cost. A weight for the weighted average may be derived based on a temporal distance between an L0 reference picture to which the L0 reference block belongs and a current picture to which the current block belongs, and a temporal distance between an L1 reference picture to which the L1 reference block belongs and the current picture. Here, the temporal distance may be defined as a difference in output order (picture order count, POC).

[0111] Alternatively, a first cost between the template area of ​​the L0 reference block and the template area of ​​the current block may be calculated, and a second cost between the template area of ​​the L1 reference block and the template area of ​​the current block may be calculated. The final cost may be calculated based on the maximum or minimum value of the first cost and the second cost.

[0112] Alternatively, the combined template region may be configured by a weighted average between samples of the template region of the L0 reference block and samples of the template region of the L1 reference block, and the cost between the combined template region and the template region of the current block may be calculated. Here, the weighted average is as described above.

[0113] Alternatively, the cost may be calculated based on a template area of ​​one of the L0 reference block or the L1 reference block, and not based on a template area of ​​the other. In this case, the cost may be limited to being calculated based only on the template area of ​​the L0 reference block, or only on the template area of ​​the L1 reference block. Alternatively, a reference picture among the L0 reference picture and the L1 reference picture having a close temporal distance to the current picture may be selected, and the cost may be calculated based on a template area of ​​a reference block belonging to the selected reference picture. When a calculation method based on template matching is used, a candidate with a smaller cost may have similar motion information to the current block, and from a coding perspective, it is advantageous to assign a small candidate index to such a candidate. Therefore, the candidates may be reordered in ascending order of the calculated cost.

[0114] Calculation method based on bilateral matching (BM)

[0115] The cost based on bilateral matching may be calculated as the cost between the L0 reference block and the L1 reference block. Here, the cost may refer to the sum of absolute differences (SAD), the sum of absolute transform differences (SATD), or the sum of squared errors (SSE).

[0116] The L0 / L1 reference block may be designated based on the motion information of a candidate in a candidate list belonging to the current block. For example, within a candidate reference picture, a block at a position shifted from the position of the current block by the candidate L0 motion vector may be determined as an L0 reference block. Within a candidate reference picture, a block at a position shifted from the position of the current block by the candidate L1 motion vector may be determined as an L1 reference block. Alternatively, within a candidate reference picture, a block at a position shifted from the position of a spatial / temporal neighboring block corresponding to the candidate by the candidate L0 motion vector may be determined as an L0 reference block. Within a candidate reference picture, a block at a position shifted from the position of a spatial / temporal neighboring block corresponding to the candidate by the candidate L1 motion vector may be determined as an L1 reference block.

[0117] The cost based on bilateral matching may be calculated based on satisfying a predetermined bilateral prediction condition. For example, the bilateral prediction condition may include at least one of the following conditions 1 to 5.

[0118] (Condition 1) The candidate motion information is motion information used for bidirectional prediction.

[0119] (Condition 2) One of the L0 reference picture to which the L0 reference block belongs and the L1 reference picture to which the L1 reference block belongs is temporally before the current picture, and the other is temporally after the current picture. That is, Condition 2 is related to the temporal order of the candidate L0 reference picture and the L1 reference picture. Here, the temporal order may refer to the output order (picture order count, POC). For example, the POC of one of the L0 reference picture and the L1 reference picture is less than the POC of the current picture, and the POC of the other is greater than the POC of the current picture.

[0120] (Condition 3) The temporal distance between the current picture and the L0 reference picture is the same as the temporal distance between the current picture and the L1 reference picture. Here, the temporal distance may refer to the difference in output order (Picture Order Count, POC).

[0121] (Condition 4) The candidate L0 / L1 reference picture does not use weighted prediction. Alternatively, the weight for weighted prediction applied to the candidate L0 reference picture is different from the weight for weighted prediction applied to the candidate L1 reference picture.

[0122] (Condition 5) The size of the candidate L0 / L1 reference picture is the same as the size of the current picture.

[0123] When using a bilateral matching-based computation method, candidates with smaller costs are likely to have similar motion information to the current block. From a coding perspective, it is advantageous to assign small candidate indices to such candidates. Therefore, the candidates can be reordered in ascending order of computation cost.

[0124] Computational methods based on motion information

[0125] The motion information-based cost may be calculated as the similarity between the motion information of the candidates in the candidate list and the motion information of the reference positions spatially / temporally adjacent to the current block.

[0126] The motion information may exist in a reference block that belongs to an area that is spatially / temporally adjacent to the current block and has a predetermined size. The similarity may be calculated by comparing the motion information of the reference block that is spatially / temporally adjacent to the current block with the motion information of the candidate belonging to the candidate list. Here, the reference block may refer to a coding / decoding unit such as a coding block, a prediction block, or a transform block. Alternatively, the reference block may be a block of a predetermined size regardless of the size of the coding / decoding unit. In this case, the block of a predetermined size may refer to a block of a predefined size such as 4x4, 8x8, or 16x16, and may be a unit that stores motion information as a block of a predefined size such as 4x4, 8x8, or 16x16.

[0127] For example, a motion information buffer may be configured in the form of a one-dimensional array based on motion information of a reference block that is spatially / temporally adjacent to the current block. A total of K motion vector fields (MVFs) may be configured in the motion information buffer. The cost between the motion vector of a candidate in the candidate list and the K motion vector fields may be calculated as shown in the following equation 1.

[0128] [Equation 1]

[0129]

[0130] In Equation 1, cost(n) represents the cost of the nth candidate in the candidate list of the current block. n represents the nth candidate motion vector, and mvf k represents the kth motion vector field. Eq(mvp n ,mvf k ) is a function used to calculate the similarity between a candidate motion vector and a motion vector field and can be defined as in Equation 2 below.

[0131] [Equation 2]

[0132]

[0133] Referring to Equation 2, when the motion vector of the nth candidate is the same as the kth motion vector field, a predefined value ∂ may be output. The value of ∂ may be a positive real number or an integer. On the other hand, when the motion vector of the nth candidate is not the same as the kth motion vector field, a predefined value 0 may be output.

[0134] Alternatively, eq(mvp n ,mvf k ) can be defined as the following Equation 3.

[0135] [Equation 3]

[0136]

[0137] Referring to Equation 3, when the motion vector of the nth candidate and the kth motion vector field are the same within a predefined accuracy, a predefined value ∂ may be output. The value of ∂ may be a positive real number or an integer. Otherwise, a predefined value of 0 may be output. In Equation 3, shift is an integer such as 1, 2, 3, or 4, and shift may vary based on the size of the block. offset is a circular offset according to the shift, and may be an integer having a value such as 1, 2, 4, or 8 based on the shift.

[0138] Alternatively, the process of configuring the motion information buffer in the form of a one-dimensional array based on the motion information of the spatial / temporal adjacent reference blocks can be omitted. In this case, the cost can be calculated as in the following equation 4 by using a motion information buffer in the form of a two-dimensional array configured around the current block.

[0139] [Equation 4]

[0140]

[0141] In Equation 4, cost(n) represents the cost of the nth candidate in the candidate list of the current block. i represents the index of the reference block in the x-axis direction located at the top of the current block, and j represents the index of the reference block in the y-axis direction located at the left side of the current block. eq x (mvp n ,mvf i ) and eq y (mvp n ,mvf j ) is a function for calculating the similarity between the nth candidate motion vector and the motion vector field, and can be defined as in the following Equations 5 and 6, respectively.

[0142] [Equation 5]

[0143]

[0144] [Equation 6]

[0145]

[0146] Referring to Equation 5, when the motion vector of the nth candidate is the same as the i-th motion vector field, a predefined value ∂ may be output. The value of ∂ may be a positive real number or an integer. On the other hand, when the motion vector of the nth candidate is different from the i-th motion vector field, a predefined value 0 may be output. Similarly, referring to Equation 6, when the motion vector of the nth candidate is the same as the j-th motion vector field, a predefined value ∂ may be output. The value of ∂ may be a positive real number or an integer. On the other hand, when the motion vector of the nth candidate is different from the j-th motion vector field, a predefined value 0 may be output.

[0147] Alternatively, eq 4 x (mvp n ,mvf i ) and eq y (mvp n ,mvf j ) can be defined as in Equations 7 and 8 below.

[0148] [Equation 7]

[0149]

[0150] [Equation 8]

[0151]

[0152] Referring to Equation 7, when the motion vector of the nth candidate and the i-th motion vector field are the same within the predefined precision, a predefined value ∂ may be output. Otherwise, a predefined value 0 may be output. The value of ∂ may be a positive real number or an integer. Similarly, referring to Equation 8, when the motion vector of the nth candidate and the j-th motion vector field are the same within the predefined precision, a predefined value ∂ may be output. Otherwise, a predefined value 0 may be output. In Equations 7 and 8, shift is an integer such as 1, 2, 3, or 4, and shift may vary based on the size of the block. Offset is an offset according to rounding of the shift, and may be an integer having a value such as 1, 2, 4, or 8 based on the shift.

[0153] When using a motion information-based calculation method, a candidate with a large cost may have similar motion information to the current block. From a coding perspective, it is advantageous to assign a small candidate index to such a candidate. Therefore, the candidates can be reordered in descending order of the calculated cost.

[0154] At least one of the above cost calculation methods can be selectively used based on the coding information of the current block. Here, the coding information may include the size of the current block, whether the current block is a block coded in sub-block mode, motion information of the candidates belonging to the candidate list, whether weighted prediction is applied to the current block, and whether scaling (or resampling) is performed on the reference block for resolution adjustment. At least one of the above-mentioned cost calculation methods can be used selectively based on the coding information of the current block. Here, the size of the current block can be defined as the width, height, maximum or minimum value of the width and height, sum of the width and height, or product of the width and height of the current block. This can be applied to the embodiments described later. When the conditions based on the coding information mentioned above are met, the cost can be calculated based on bilateral matching, and otherwise, the cost can be calculated based on template matching.

[0155] At least one of the above cost calculation methods can be selectively used based on whether the current block meets the block size condition. When the current block meets the block size condition, the cost can be calculated based on bilateral matching. On the other hand, when the current block does not meet the block size condition, the cost can be calculated based on template matching.

[0156] For example, when the size of the current block is less than a threshold, the cost may be calculated based on bilateral matching. On the other hand, when the size of the current block is greater than or equal to the threshold, the cost may be calculated based on template matching.

[0157] When the size of the current block is defined as the width or height of the current block, the threshold value may be an integer of 8, 16, 32 or more. When the size of the current block is defined as the product of the width and height of the current block, the threshold value may be an integer of 64, 128, 256 or more.

[0158] For example, when the product of the width and height of the current block is less than 128, the cost may be calculated based on bilateral matching. On the other hand, when the product of the width and height of the current block is greater than or equal to 128, the cost may be calculated based on template matching.

[0159] For example, when the size of the current block is greater than a threshold, the cost may be calculated based on bilateral matching. On the other hand, when the size of the current block is less than or equal to the threshold, the cost may be calculated based on template matching.

[0160] When the size of the current block is defined as the width or height of the current block, the threshold value may be an integer of 8, 16, 32 or more. When the size of the current block is defined as the product of the width and height of the current block, the threshold value may be an integer of 64, 128, 256 or more.

[0161] For example, when the product of the width and height of the current block is greater than 128, the cost may be calculated based on bilateral matching. On the other hand, when the product of the width and height of the current block is less than or equal to 128, the cost may be calculated based on template matching.

[0162] At least one of the above cost calculation methods may be selectively used based on whether the candidate of the current block satisfies the above bidirectional prediction condition. Here, the bidirectional prediction condition may include at least one of conditions 1 to 5 as described above, and a detailed description thereof will be omitted here.

[0163] For example, when the candidate of the current block meets the bidirectional prediction condition, the cost of the candidate can be calculated based on bilateral matching. On the other hand, when the candidate of the current block does not meet the bidirectional prediction condition, the cost of the candidate can be calculated based on template matching.

[0164] At least one of the above cost calculation methods may be selectively used based on whether the current block is a block coded in sub-block mode. Sub-block mode may refer to a mode in which the current block is divided into a plurality of sub-blocks and inter-frame prediction is performed in units of sub-blocks. When the current block is coded in sub-block mode, the current block may be divided into sub-blocks of a predetermined size, and each sub-block may be decoded using different motion information. Here, different motion information may mean that at least one of a motion vector, a reference picture index, inter-frame prediction direction information, or weight information for bidirectional weighted prediction is different. For example, multiple sub-blocks may share the same reference picture index or inter-frame prediction direction information, but may have different motion vectors or weight information. The size of the sub-block may be a predefined fixed size, such as 4x4, 8x8, or 16x16, or may be variably determined based on the size of the current block.

[0165] For example, when the current block is a block coded in the sub-block mode, the cost may be calculated based on bilateral matching. On the other hand, when the current block is not a block coded in the sub-block mode, the cost may be calculated based on template matching.

[0166] At least one of the above cost calculation methods may be selectively used based on the position information of the current block. Specifically, when the current block is located at a position where the template region and / or the reference block of the current block is not configured (or where the template region and / or the reference block of the current block does not exist), the cost may be calculated based on bilateral matching, and otherwise, the cost may be calculated based on template matching.

[0167] The case where the current block exists in the template area and / or the location of the reference block where the current block is not configured may mean the case where a boundary of the current block is located at a boundary of a picture, a slice, a tile, a virtual pipeline decoding unit (VPDU), a coding tree unit (CTU) row, or a CTU or is adjacent to a boundary of a picture, a slice, a tile, a virtual pipeline decoding unit (VPDU), a coding tree unit (CTU) row, or a CTU.

[0168] At least one of the above cost calculation methods can be selectively used based on at least two of whether the current block satisfies a block size condition, whether a candidate of the current block satisfies a bidirectional prediction condition, whether the current block is a block coded in a sub-block mode, or position information of the current block.

[0169] For example, when the current block meets the block size condition and the candidate of the current block meets the bidirectional prediction condition, the cost can be calculated based on bilateral matching. When the current block meets the block size condition but the candidate of the current block does not meet the bidirectional prediction condition, the cost can be calculated based on template matching. When the current block does not meet the block size condition, the cost can be calculated based on template matching. When the current block does not meet the block size condition, regardless of whether the candidate of the current block meets the bidirectional prediction condition, the cost can be calculated based on template matching.

[0170] For example, when the current block is a block coded in sub-block mode and the candidate of the current block meets the bidirectional prediction condition, the cost can be calculated based on bilateral matching. When the current block is a block coded in sub-block mode but the candidate of the current block does not meet the bidirectional prediction condition, the cost can be calculated based on template matching. When the current block is not a block coded in sub-block mode, the cost can be calculated based on template matching. When the current block is not a block coded in sub-block mode, regardless of whether the candidate of the current block meets the bidirectional prediction condition, the cost can be calculated based on template matching.

[0171] For example, when the current block satisfies the block size condition or is a block coded in the sub-block mode, when the candidate of the current block satisfies the bidirectional prediction condition, the cost may be calculated based on bilateral matching. When the current block satisfies the block size condition or is a block coded in the sub-block mode, when the candidate of the current block does not satisfy the bidirectional prediction condition, the cost may be calculated based on template matching. When the current block does not satisfy the block size condition and is not a block coded in the sub-block mode, the cost may be calculated based on template matching. When the current block does not satisfy the block size condition and is not a block coded in the sub-block mode, the cost may be calculated based on template matching regardless of whether the candidate of the current block satisfies the bidirectional prediction condition.

[0172] For example, when the current block exists at a position of a template region and / or a reference block where the current block is not configured, a calculation method based on bilateral matching may be adaptively used based on whether a candidate of the current block satisfies a bidirectional prediction condition.

[0173] For example, in the case where the current block exists in a template area and / or a reference block where the current block is not configured, when the candidates of the current block meet the bidirectional prediction conditions, the cost can be calculated based on bilateral matching, and when the candidates of the current block do not meet the bidirectional prediction conditions, the candidate list of the current block may not be reordered.

[0174] On the other hand, when the current block exists at a position of a template area and / or a reference block in which the current block can be configured, one of a calculation method based on bilateral matching or a calculation method based on template matching can be adaptively used based on whether the current block satisfies a block size condition, whether the current block is a block coded in a sub-block mode, or whether a candidate of the current block satisfies at least one of a bidirectional prediction condition.

[0175] For example, when the current block exists at a location of a template area and / or a reference block in which the current block can be configured, when the current block satisfies a block size condition or is a block coded in a sub-block mode and a candidate of the current block satisfies a bidirectional prediction condition, the cost can be calculated based on bilateral matching.

[0176] However, in the case where the current block exists at a position of a template area and / or a reference block in which the current block can be configured, when the candidate of the current block does not satisfy the bidirectional prediction condition, the cost can be calculated based on template matching even if the current block satisfies the block size condition or is not a block coded in the sub-block mode.

[0177] Even if the current block exists at a location where a template region and / or a reference block can be configured for the current block, when the current block does not satisfy the block size condition and is not a block coded in the sub-block mode, the cost may be calculated based on template matching. Even if the current block exists at a location where a template region and / or a reference block can be configured for the current block, when the current block does not satisfy the block size condition and is not a block coded in the sub-block mode, the cost may be calculated based on template matching regardless of whether the candidate of the current block satisfies the bidirectional prediction condition.

[0178] In the case where one of the above cost calculation methods is selectively used for each candidate in the candidate list, the cost of the candidate can be calculated based on different cost calculation methods. That is, the cost of one of the candidates can be calculated based on template matching, and the cost of another candidate can be calculated based on bilateral matching. In such a case, the calculated costs can be normalized and the candidate list can be reordered based on the normalized costs.

[0179] For example, when the cost is calculated based on template matching, the cost can be normalized based on the number of pixels belonging to the template area or reference block of the current block. When the cost is calculated based on bilateral matching, the cost can be normalized based on the number of pixels belonging to the L0 reference block or the L1 reference block.

[0180] All pixels in the target block may be used to calculate the cost. Alternatively, some pixels in the target block may be used to calculate the cost, thereby reducing the complexity of the cost calculation. Here, the target block may mean a template region and / or a reference block of the current block used for cost calculation based on template matching. Alternatively, the target block may mean an L0 / L1 reference block used for cost calculation based on bilateral matching.

[0181] Some pixels of the target block may be determined as pixels belonging to a sub-sampled pixel column in the target block.

[0182] For example, the sub-sampled pixel columns may mean a set of even-numbered pixel columns. Assuming that the coordinates of the upper left corner pixel of the target block are (x, y), the set of even-numbered pixel columns may include at least one of the following: the 0th pixel column including pixels of (x, y), the 2nd pixel column including pixels of (x, y+2), or the 4th pixel column including pixels of (x, y+4). Alternatively, by increasing the unit of sub-sampling, a set of even-numbered pixel columns including at least one of the following may be used, including the 0th pixel column including pixels of (x, y), the 4th pixel column including pixels of (x, y+4), or the 8th pixel column including pixels of (x, y+8).

[0183] For example, the sub-sampled pixel columns may mean odd-numbered pixel columns. Assuming that the coordinates of the upper left corner pixel of the target block are (x, y), the set of odd-numbered pixel columns may include at least one of the following: the 1st pixel column including pixels of (x, y+1), the 3rd pixel column including pixels of (x, y+3), or the 5th pixel column including pixels of (x, y+5). Alternatively, by increasing the unit of sub-sampling, a set of odd-numbered pixel columns including at least one of the following may be used: the 1st pixel column including pixels of (x, y+1), the 5th pixel column including pixels of (x, y+5), or the 9th pixel column including pixels of (x, y+9).

[0184] The above-described embodiment uses pixel columns within the target block, but is not limited thereto, and of course, pixel rows may be used instead of pixel columns within the target block.

[0185] Some pixels of the target block may be determined as pixels belonging to sub-sampled pixel columns and sub-sampled pixel rows within the target block. For example, some pixels of the target block may be determined as a set of pixels belonging to sub-sampled pixel columns and pixels belonging to sub-sampled pixel rows within the target block. Alternatively, some pixels of the target block may be determined as pixels belonging to sub-sampled pixel rows while belonging to sub-sampled pixel columns within the target block. Sub-sampled pixel columns and sub-sampled pixel rows are as described above, and a repeated description thereof will be omitted herein.

[0186] Some pixels of the target block may be determined as pixels belonging to a sub-sampled sub-block within the target block. For example, the sub-sampled sub-block may mean a set of sub-blocks having an even scan index according to a predetermined scan order. Alternatively, the sub-sampled sub-block may mean a set of sub-blocks having an odd scan order according to a predetermined scan order. Here, the scan order may be any one of a raster scan order, a diagonal scan order, a horizontal scan order, or a vertical scan order.

[0187] Some pixels of the target block may be determined as pixels of some sub-blocks among the sub-sampled sub-blocks. For example, some pixels of the target block may mean pixels of the first N sub-blocks in ascending order of the scan index of the sub-block for sub-sampling. Here, N may be an integer of 1, 2, 3, 4 or more.

[0188] The target block may be divided into a plurality of sub-blocks having a predetermined size. In this case, some pixels of the target block may be determined as pixels belonging to the first N sub-blocks in ascending order of the scan index according to a predetermined scan order among the sub-blocks in the target block. Here, N may be an integer of 1, 2, 3, 4 or more. Alternatively, some pixels of the target block may be determined as pixels belonging to a sub-block in a specific direction among the sub-blocks in the target block. Here, the specific direction may mean an upper left direction, a left direction or an upper direction.

[0189] Some pixels of the target block may be determined by a combination of the above-mentioned embodiments related to the sub-sampled pixel columns and / or pixel rows and the above-mentioned embodiments related to the sub-sampled sub-blocks. That is, some pixels of the target block may be determined as pixels belonging to the sub-sampled pixel columns and / or pixel rows within the sub-block according to the above-mentioned embodiments.

[0190] The cost for reordering the candidate list may be calculated for all candidates in the candidate list, or may be calculated for some of the candidates in the candidate list. For example, some candidates may be limited to the first M candidates in ascending order of the candidate index assigned to the candidate list. Here, M may be an integer of 2, 3, 4 or more. The value of M may be a predefined value regardless of the maximum number of candidates to be included in the candidate list. Alternatively, some candidates may be limited to spatial candidates and / or temporal candidates among the candidates in the candidate list. In this way, the complexity of calculating the cost for reordering the candidate list may be reduced by calculating the cost only for some of the candidates in the candidate list instead of all the candidates.

[0191] It can be determined whether to calculate costs for all candidates in the candidate list based on whether the maximum number of candidates to be included in the candidate list is less than a predetermined threshold number. Here, the threshold number can be an integer such as 4, 5, 6, 7, 8, or greater. For example, when the maximum number of candidates to be included in the candidate list is less than or equal to the threshold number, costs are calculated for all candidates in the candidate list. Otherwise, costs can be calculated for some of the above candidates.

[0192] Reference Figure 4 , the motion information of the current block can be derived based on the candidate list and the candidate index (S420).

[0193] The candidate index can mean information compiled to predict / derive the motion information of the current block. The candidate index can specify one or more of multiple candidates belonging to the candidate list.

[0194] The motion vector of the motion information can mean a motion vector in units of blocks. Alternatively, when the current block is a block compiled in the sub - block mode, the motion vector of the motion information can mean a motion vector derived in units of the sub - blocks of the current block. For this purpose, as described above, the current block can be divided into multiple NxM sub - blocks. Here, the NxM sub - blocks can have a rectangular (N > M or N < M) or square (N = M) shape. The values of N and M can be 2, 4, 8, 16, 32, or greater.

[0195] Reference Figure 4 , inter - frame prediction can be performed on the current block based on the derived motion information (S430).

[0196] Specifically, the motion information of the current block can be used to specify a reference block. When the motion information of the current block is motion information for bidirectional prediction, a reference block can be specified for each of the L0 direction and the L1 direction.

[0197] When the current block is a block compiled in the sub - block mode, a reference block can be specified for each sub - block of the current block. The reference blocks of each sub - block can belong to one reference picture. That is, the sub - blocks belonging to the current block can share one reference picture. Alternatively, a reference picture index can be independently set for each sub - block of the current block.

[0198] A predicted block of the current block can be generated based on the specified reference block.

[0199] Figure 5 Schematic configuration of an inter - frame predictor (332) performing an inter - frame prediction method according to the present disclosure is illustrated.

[0200] Reference Figure 4 Describes an inter - frame prediction method performed in a decoding device. This can be performed in the same way in the inter - frame predictor (332) of the decoding device, and its detailed description will be omitted below.

[0201] refer to Figure 5 , the inter-frame predictor (332) may include a candidate list generator (500), a motion information deriver (510), and a prediction sample generator (520).

[0202] The candidate list generator (500) may generate a candidate list for predicting / deriving motion information of a current block. The candidate list may be used for a merge mode or an AMVP mode, or may be used for an affine merge mode or an affine inter mode. The motion information may include at least one of a motion vector, a reference picture index, inter prediction direction information, or weight information for bidirectional weighted prediction.

[0203] The candidate list generator (500) may derive a plurality of candidates including at least one of spatial candidates or temporal candidates, and generate a candidate list based on the plurality of candidates.

[0204] The candidate list generator (500) may reorder one or more candidates in the candidate list based on a predetermined cost. That is, a cost may be calculated for each candidate in the candidate list, and the candidates may be reordered in ascending or descending order of the calculated cost to generate a final candidate list.

[0205] The candidate list generator (500) may calculate the cost for reordering the candidate list by using at least one of a calculation method based on template matching (TM), a calculation method based on bilateral matching (BM), or a calculation method based on motion information, as shown in FIG. Figure 4 described.

[0206] The candidate list generator (500) can calculate the cost for reordering the candidate list by selecting at least one of the above cost calculation methods based on the coding information of the current block. For example, at least one of the above cost calculation methods can be selectively used based on at least one of whether the current block satisfies the block size condition, whether the candidate of the current block satisfies the bidirectional prediction condition, whether the current block is a block coded in the sub-block mode, or the position information of the current block, as shown in reference Figure 4 described.

[0207] When one of the above cost calculation methods is selectively used for each candidate in the candidate list, the cost of the candidate can be calculated based on different cost calculation methods. In this case, the candidate list generator (500) can perform normalization on the calculated costs and re-rank the candidate list based on the normalized costs.

[0208] All pixels of the target block to be used for cost calculation can be used to calculate the cost for reordering the candidate list. Alternatively, some pixels within the target block can be used for the calculation, and for this purpose, the candidate list generator (500) can determine some pixels within the target block for cost calculation. The method for determining some pixels within the target block is as Figure 4 shown.

[0209] As described above, the cost for reordering the candidate list can be calculated for all candidates in the candidate list or for some candidates in the candidate list.

[0210] The motion information exporter (510) can export the motion information of the current block based on the candidate list and the candidate index. Here, the candidate index is the information encoded to predict / export the motion information of the current block, and can specify one or more candidates among the multiple candidates in the candidate list.

[0211] When exporting the motion information of the current block, the motion information exporter (510) can export the motion vector in units of blocks, or can export the motion vector in units of sub-blocks of the current block. To export the motion vector in units of sub-blocks, the motion information exporter (510) can divide the current block into units of NxM sub-blocks. Here, the NxM sub-blocks can have a rectangular (N>M or N<M) or square (N = M) shape. The values of N and M can be 2, 4, 8, 16, 32 or larger.

[0212] The prediction sample generator (520) can perform inter-frame prediction based on the motion information obtained from the motion information exporter (510) to generate the prediction samples of the current block.

[0213] Specifically, the prediction sample generator (520) can specify the reference block using the motion information of the current block, and can generate the prediction block of the current block based on the specified reference block.

[0214] Alternatively, when the motion vector is exported in units of sub-blocks in the motion information exporter (510), the prediction sample generator (520) can specify the reference block for each sub-block of the current block, and generate the prediction block of the current block based on the specified reference block. In this case, the specified reference block can belong to one reference picture. Alternatively, the reference picture index can be independently set for each sub-block of the current block, and in this case, one of the specified reference blocks can belong to a different reference picture from another.

[0215] Figure 6 The figure illustrates an inter-frame prediction method performed by an encoding device according to an embodiment of the present disclosure.

[0216] Reference Figure 6, a candidate list for determining motion information of a current block may be generated ( S600 ).

[0217] The motion information may include at least one of a motion vector, a reference picture index, inter prediction direction information, or weight information for bidirectional weighted prediction. The candidate list may be used for merge mode or AMVP mode, or may be used for affine merge mode or affine inter mode.

[0218] A plurality of candidates including at least one of the spatial candidates or the temporal candidates may be derived, and a candidate list may be generated based on the derived plurality of candidates.

[0219] refer to Figure 6 , one or more candidates in the candidate list may be reordered based on a predetermined cost ( S610 ).

[0220] Reordering according to the present disclosure may refer to changing the position of a candidate in a candidate list or changing a candidate index assigned to a candidate. To this end, a cost for reordering may be calculated for each candidate in the candidate list. The candidates may be reordered in ascending or descending order of the calculated cost to generate a final candidate list.

[0221] The cost according to the present disclosure may be calculated using at least one of a calculation method based on template matching (TM), a calculation method based on bilateral matching (BM), or a calculation method based on motion information. Figure 4 described.

[0222] In addition, at least one of the above cost calculation methods may be selectively used to calculate the cost for reordering the candidate list based on the coding information about the current block. For example, at least one of the above cost calculation methods may be selectively used based on whether the current block satisfies the block size condition, whether the candidate of the current block satisfies the bidirectional prediction condition, whether the current block is a block coded in a sub-block mode, or at least one of the position information of the current block, as shown in FIG. Figure 4 described.

[0223] In the case where one of the above cost calculation methods is selectively used for each candidate in the candidate list, the cost of the candidate can be calculated based on different cost calculation methods. In this case, the calculated costs can be normalized, and the candidate list can be reordered based on the normalized costs.

[0224] The cost for reordering the candidate list may be calculated using all pixels of the target block to be used for cost calculation, or may be calculated using some pixels within the target block. A method for determining some pixels within the target block for cost calculation is as follows: Figure 4 shown.

[0225] As described above, the cost for re-ordering the candidate list may be calculated for all candidates in the candidate list or for some of the candidates in the candidate list.

[0226] refer to Figure 6 , inter-frame prediction may be performed on the current block based on the candidate list ( S620 ).

[0227] A prediction sample of the current block may be generated based on at least one candidate among a plurality of candidates belonging to the candidate list.

[0228] Specifically, the reference block may be specified using motion information of the best candidate among multiple candidates. A prediction block of the current block may be generated based on the specified reference block. When the current block is encoded in sub-block mode, a reference block may be specified for each sub-block of the current block, and a prediction block of the current block may be generated based on the specified reference block. The reference block corresponding to the sub-block of the current block may belong to one reference picture. Alternatively, a reference picture index may be independently set for each sub-block of the current block, and in this case, one of the specified reference blocks may belong to a reference picture different from another.

[0229] Among the plurality of candidates belonging to the candidate list, at least one best candidate for generating a prediction sample of the current block may be determined. A candidate index specifying the determined candidate may be encoded and inserted into a bitstream.

[0230] Figure 7 The diagram illustrates a schematic configuration of an inter-frame predictor (221) that performs an inter-frame prediction method according to the present disclosure.

[0231] refer to Figure 6 The inter-frame prediction method performed in the encoding device is described. This can be performed in the same manner in the inter-frame predictor (221) of the encoding device, and its detailed description will be omitted below.

[0232] refer to Figure 7 , the inter-frame predictor (221) may include a candidate list generator (700) and a prediction sample generator (710).

[0233] The candidate list generator (700) may generate a candidate list for determining motion information of a current block. The motion information may include at least one of a motion vector, a reference picture index, inter prediction direction information, or weight information for bidirectional weighted prediction. The candidate list may be used for merge mode or AMVP mode, or for affine merge mode or affine inter mode.

[0234] The candidate list generator (700) may derive a plurality of candidates including at least one of spatial candidates or temporal candidates, and generate a candidate list based on the plurality of candidates.

[0235] The candidate list generator (700) may reorder one or more candidates in the candidate list based on a predetermined cost. That is, a cost may be calculated for each candidate in the candidate list, and the candidates may be reordered in ascending or descending order of the calculated cost to generate a final candidate list.

[0236] The candidate list generator (700) may calculate the cost for reordering the candidate list by using at least one of a calculation method based on template matching (TM), a calculation method based on bilateral matching (BM), or a calculation method based on motion information, as shown in FIG. Figure 4 described.

[0237] The candidate list generator (700) can calculate the cost for reordering the candidate list by selecting at least one of the above cost calculation methods based on the coding information of the current block. For example, at least one of the above cost calculation methods can be selectively used based on at least one of whether the current block satisfies the block size condition, whether the candidate of the current block satisfies the bidirectional prediction condition, whether the current block is a block coded in the sub-block mode, or the position information of the current block, as shown in reference Figure 4 described.

[0238] If one of the above cost calculation methods is selectively used for each candidate in the candidate list, the cost of the candidate can be calculated based on different cost calculation methods. In this case, the candidate list generator (700) can perform normalization on the calculated costs and re-rank the candidate list based on the normalized costs.

[0239] The cost for reordering the candidate list may be calculated using all pixels of the target block to be used for cost calculation. Alternatively, some pixels within the target block may be used for calculation, and for this purpose, the candidate list generator (700) may determine some pixels within the target block for cost calculation. The method for determining some pixels within the target block is as follows: Figure 4 shown.

[0240] As described above, the cost for re-ordering the candidate list may be calculated for all candidates in the candidate list or for some of the candidates in the candidate list.

[0241] The prediction sample generator (710) may perform inter-frame prediction on the current block based on the candidate list. The prediction sample of the current block may be generated based on at least one best candidate among a plurality of candidates included in the candidate list.

[0242] Specifically, the prediction sample generator (710) may specify a reference block using motion information of a best candidate among a plurality of candidates, and may generate a prediction block of the current block based on the specified reference block. When the current block is encoded in a sub-block mode, a reference block may be specified for each sub-block of the current block, and a prediction block of the current block may be generated based on the specified reference block. In this case, the reference blocks corresponding to the sub-blocks of the current block may belong to one reference picture. Alternatively, a reference picture index may be independently set for each sub-block of the current block, and in this case, one of the specified reference blocks may belong to a reference picture different from another.

[0243] In addition, the prediction sample generator (710) may further include a motion information determiner (not shown). The motion information determiner may determine at least one candidate for generating a prediction sample of the current block from among a plurality of candidates belonging to the candidate list. In this case, the entropy encoder (240) may encode a candidate index specifying the determined candidate and insert it into the bitstream.

[0244] In the above embodiments, the method is described as a series of steps or boxes based on the flowchart, but the corresponding embodiments are not limited to the order of the steps, and some steps may occur simultaneously or in a different order than other steps described above. In addition, those skilled in the art will appreciate that the steps shown in the flowchart are not exclusive, and other steps may be included or one or more steps in the flowchart may be deleted without affecting the scope of the embodiments of the present disclosure.

[0245] The above-mentioned method according to the embodiment of the present disclosure can be implemented in the form of software, and the encoding device and / or decoding device according to the present disclosure can be included in a device that performs image processing, such as a TV, a computer, a smart phone, a set-top box, a display device, etc.

[0246] In the present disclosure, when the embodiment is implemented as software, the above method can be implemented as a module (process, function, etc.) that performs the above functions. The module can be stored in a memory and can be executed by a processor. The memory can be located inside or outside the processor and can be connected to the processor by various well-known means. The processor may include an application-specific integrated circuit (ASIC), another chipset, a logic circuit, and / or a data processing device. The memory may include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium, and / or other storage devices. In other words, the embodiments described herein can be implemented on a processor, a microprocessor, a controller, or a chip. For example, the functional unit shown in each of the figures can be implemented on a computer, a processor, a microprocessor, a controller, or a chip. In this case, information (e.g., information about instructions) or an algorithm for implementation can be stored in a digital storage medium.

[0247] In addition, the decoding device and the encoding device of the embodiment of the present disclosure can be included in multimedia broadcast sending and receiving devices, mobile communication terminals, home theater video devices, digital theater video devices, surveillance cameras, video conversation devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, devices for providing video on demand (VoD) services, over-the-top video (OTT) devices, devices for providing Internet streaming services, three-dimensional (3D) video devices, virtual reality (VR) devices, augmented reality (AR) devices, videophone video devices, transportation terminals (e.g., vehicle (including autonomous driving vehicle) terminals, aircraft terminals, ship terminals, etc.) and medical video devices, etc., and can be used to process video signals or data signals. For example, over-the-top video (OTT) devices may include game consoles, Blu-ray players, networked TVs, home theater systems, smart phones, tablet computers, digital video recorders (DVRs), etc.

[0248] In addition, the processing method of the embodiment of the present disclosure can be generated in the form of a program executed by a computer and can be stored in a computer-readable recording medium. Multimedia data having a data structure according to an embodiment of the present disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices and distributed storage devices that store computer-readable data. Computer-readable recording media may include, for example, Blu-ray discs (BD), universal serial buses (USB), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, tapes, floppy disks, and optical media storage devices. In addition, computer-readable recording media include media implemented in the form of carrier waves (e.g., transmitted via the Internet). In addition, the bit stream generated by the encoding method may be stored in a computer-readable recording medium or may be sent via a wired or wireless communication network.

[0249] In addition, the embodiments of the present disclosure may be implemented by a computer program product through a program code, and the program code may be executed on a computer by the embodiments of the present disclosure. The program code may be stored on a computer-readable carrier.

[0250] Figure 8 An example of a content streaming system to which an embodiment of the present disclosure can be applied is shown.

[0251] refer to Figure 8 The content streaming transmission system to which the embodiments of the present disclosure are applied may mainly include an encoding server, a streaming transmission server, a web server, a media storage, a user device, and a multimedia input device.

[0252] The encoding server generates a bitstream by compressing content input from a multimedia input device such as a smartphone, a camera, a camcorder, etc. into digital data and transmits it to the streaming server. As another example, when a multimedia input device such as a smartphone, a camera, a camcorder, etc. directly generates a bitstream, the encoding server may be omitted.

[0253] A bitstream may be generated by applying the encoding method or the bitstream generating method of the embodiment of the present disclosure, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0254] The streaming server sends multimedia data to the user device through the web server based on the user's request, and the web server serves as a medium to inform the user what services are available. When the user requests the required service from the web server, the web server delivers it to the streaming server, and the streaming server sends the multimedia data to the user. In this case, the content streaming system may include a separate control server, and in this case, the control server controls the command / response between each device in the content streaming system.

[0255] The streaming server may receive content from a media storage and / or encoding server. For example, when receiving content from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bitstream for a certain period of time.

[0256] Examples of user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation, tablet PCs, tablet computers, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays (HMDs), digital televisions, desktops, digital signage, etc.).

[0257] Each server in the content streaming system may be operated as a distributed server, and in this case, data received from each server may be distributed and processed.

[0258] The claims set forth herein can be combined in various ways. For example, the technical features of the method claims of the present disclosure can be combined and implemented as a device, and the technical features of the device claims of the present disclosure can be combined and implemented as a method. In addition, the technical features of the method claims of the present disclosure and the technical features of the device claims can be combined and implemented as a device, and the technical features of the method claims of the present disclosure and the technical features of the device claims can be combined and implemented as a method.

Claims

1. An image decoding method, comprising: Generate a candidate list for the current block; reordering one or more candidates in the candidate list based on a predetermined cost, and calculating the cost for each of the plurality of candidates in the candidate list according to at least one of a calculation method based on template matching or a calculation method based on bilateral matching; deriving motion information of the current block based on the candidate list and the candidate index; as well as Based on the motion information, inter-frame prediction is performed on the current block.

2. The image decoding method according to claim 1, wherein: One of the calculation method based on template matching or the calculation method based on bilateral matching is selectively used based on whether the current block satisfies a block size condition.

3. The image decoding method according to claim 1, wherein: selectively using one of the calculation method based on template matching or the calculation method based on bilateral matching based on whether the candidate belonging to the candidate list satisfies the bidirectional prediction condition, and Among them, the bidirectional prediction condition includes at least one of the first condition, the second condition, the third condition, the fourth condition or the fifth condition, the first condition is that the candidate motion information is motion information for bidirectional prediction, the second condition is that one of the L0 reference picture or the L1 reference picture is temporally before the current picture and the other is temporally after the current picture, the third condition is that the temporal distance between the current picture and the L0 reference picture is the same as the temporal distance between the current picture and the L1 reference picture, the fourth condition is that the candidate L0 reference picture and the L1 reference picture use weighted prediction, and the fifth condition is that the candidate L0 reference picture and the L1 reference picture have the same size as the current picture.

4. The image decoding method according to claim 1, wherein: Based on whether the current block is a block coded in a sub-block mode, one of the calculation method based on template matching or the calculation method based on bilateral matching is selectively used.

5. The image decoding method according to claim 1, wherein: Based on the position information of the current block, one of the calculation method based on template matching or the calculation method based on bilateral matching is selectively used.

6. The image decoding method according to claim 1, wherein: Based on whether the current block satisfies a block size condition, whether the candidates in the candidate list satisfy a bidirectional prediction condition, whether the current block is a block coded in a sub-block mode, or at least two of the position information of the current block, one of the template matching-based calculation method or the bilateral matching-based calculation method is selectively used.

7. The image decoding method according to claim 6, wherein: When the current block satisfies the block size condition and the candidates in the candidate list satisfy the bidirectional prediction condition, the cost is calculated using the bilateral matching-based calculation method.

8. The image decoding method according to claim 6, wherein: When the current block is coded in the sub-block mode and the candidate satisfies the bidirectional prediction condition, the cost is calculated using the bilateral matching-based calculation method.

9. The image decoding method according to claim 6, wherein: When the current block exists at a position where a template region for the calculation method based on template matching is not configured and the candidate satisfies the bidirectional prediction condition, the cost is calculated using the calculation method based on bilateral matching.

10. The image decoding method according to claim 6, wherein: The calculation method based on bilateral matching calculates the cost using an L0 reference block and an L1 reference block specified based on motion information of a candidate belonging to the candidate list, wherein the cost is calculated based on some pixels in the L0 reference block and the L1 reference block, and The some pixels are determined as subsampled pixel columns in the L0 reference block and the L1 reference block, subsampled subblocks in the L0 reference block and the L1 reference block, or subsampled pixel columns in the subsampled subblock.

11. An image encoding method, comprising: Generate a candidate list for the current block; reordering one or more candidates in the candidate list based on a predetermined cost, and calculating the cost for each of the plurality of candidates in the candidate list according to at least one of a calculation method based on template matching or a calculation method based on bilateral matching; as well as Inter-frame prediction is performed on the current block based on the candidate list.

12. A computer-readable storage medium storing a bit stream generated by the image encoding method according to claim 11.

13. A method for transmitting data for image information, comprising: obtaining a bitstream for the image information, wherein the bitstream is generated by: generating a candidate list for a current block, reordering one or more candidates in the candidate list based on a predetermined cost, and performing inter-frame prediction on the current block based on the candidate list; as well as sending said data comprising said bit stream, The cost is calculated for each of the multiple candidates in the candidate list according to at least one of a calculation method based on template matching or a calculation method based on bilateral matching.

Images