Decoding device, encoding device, and data transmitting device

Abstract

Decoding equipment, encoding equipment, and data transmission equipment are disclosed. A method for decoding video signals and an apparatus for using the method are disclosed. Specifically, a method for decoding an image based on an inter-frame prediction mode may include the following steps: configuring a merged candidate list using motion information of spatial and temporal candidates adjacent to the current block; parsing a first syntax element indicating a specific candidate among a predefined number of candidates in the merged candidate list for inter-frame prediction of the current block; deriving a modified candidate by adding a motion vector offset to the motion vector of the candidate indicated by the first syntax element; and generating a prediction block of the current block using the motion information of the modified candidate.

Description

[0001] This application is a divisional application of the original invention patent application No. 201980045852.3 (International Application No.: PCT / KR2019 / 005668, Application Date: May 10, 2019, Invention Title: Method for Processing Images Based on Inter-Frame Prediction Mode and Apparatus for the Method). Technical Field

[0002] This disclosure relates to methods for processing still images or moving pictures, and more specifically, to methods for encoding / decoding still images or moving pictures based on inter-frame prediction modes and devices supporting such methods. Background Technology

[0003] Compression coding refers to a series of signal processing techniques used to transmit digital information over communication lines or to store information in a form suitable for storage media. Media including pictures, images, and audio can be targets of compression coding, and specifically, the techniques used to perform compression coding on pictures are called video image compression.

[0004] Next-generation video content should feature high spatial resolution, high frame rates, and high-dimensional scene rendering. Processing this content will necessitate a dramatic increase in memory storage, memory access speeds, and processing power.

[0005] Therefore, there is a need to design encoding tools for efficiently processing next-generation video content. Summary of the Invention

[0006] Technical issues

[0007] Embodiments of this disclosure provide a method for generating modified merge candidates by using merge offsets.

[0008] Furthermore, embodiments of this disclosure provide a method for generating modified merge candidates by using a weighted average.

[0009] Furthermore, embodiments of this disclosure propose a method for adding candidates by using the average of candidates selected in a merged candidate list.

[0010] The technical objectives of this disclosure are not limited to those mentioned above, and other technical objectives not mentioned above will become apparent to those skilled in the art from the following description.

[0011] Technical solution

[0012] In one aspect of this disclosure, a method for decoding an image based on an inter-frame prediction mode may include the following steps: configuring a merged candidate list by using motion information of spatial and temporal candidates adjacent to the current block; parsing a first syntax element indicating a specific candidate among a predefined number of candidates in the merged candidate list that is applied to the inter-frame prediction of the current block; deriving a modified candidate by adding a motion vector offset to the motion vector of the candidate indicated by the first syntax element; and generating a prediction block of the current block by using the motion information of the modified candidate.

[0013] Preferably, the parsing of the first syntax element can be performed by parsing the first syntax element of a specific candidate among the first and second candidates that indicate the order in which the merge indexes are assigned in the merge candidate list and which is applied to the inter-frame prediction of the current block.

[0014] Preferably, the motion vector offset can be derived based on the motion vector offset precision indicating whether the motion vector offset has integer pixel precision or fractional pixel precision.

[0015] Preferably, deriving the candidate for the correction may include resolving a second syntax element that indicates the accuracy of the motion vector offset, and the second syntax element may be sent via a sequence parameter set, an image parameter set, or a slice header.

[0016] Preferably, the motion vector offset can be derived based on a value obtained by multiplying the motion vector offset precision by a non-zero integer.

[0017] Preferably, configuring the merged candidate list may further include adding candidates generated by weighted averaging of pairs of predefined candidates in the merged candidate list to the merged candidate list.

[0018] Preferably, configuring the merge candidate list may further include adding candidates generated by averaging pairs of predefined candidates in the merge candidate list to the merge candidate list.

[0019] In another aspect of this disclosure, an apparatus for decoding an image based on an inter-frame prediction mode may include: a merge candidate list configuration unit that configures a merge candidate list using motion information of spatial and temporal candidates adjacent to the current block; a first syntax element parsing unit that parses a first syntax element indicating a specific candidate among a predefined number of candidates in the merge candidate list for inter-frame prediction of the current block; a corrected candidate derivation unit that derives corrected candidates by adding a motion vector offset to the motion vector of the candidate indicated by the first syntax element; and a prediction block generation unit that generates a prediction block of the current block using motion information of the current block.

[0020] Preferably, the first syntax element parsing unit can parse the first syntax element indicating a specific candidate among the first and second candidates that are assigned merge indices in the merge candidate list and are applied to the inter-frame prediction of the current block.

[0021] Preferably, the motion vector offset can be derived based on the motion vector offset precision indicating whether the motion vector offset has integer pixel precision or fractional pixel precision.

[0022] Preferably, the correction candidate derivation unit can parse a second syntax element indicating the accuracy of the motion vector offset, and the second syntax element can be sent through a sequence parameter set, an image parameter set, or a slice header.

[0023] Preferably, the motion vector offset can be derived based on a value obtained by multiplying the motion vector offset precision by a non-zero integer.

[0024] Preferably, the merged candidate list configuration unit can add candidates generated by weighted averaging of pairs of predefined candidates in the merged candidate list to the merged candidate list.

[0025] Preferably, the merged candidate list configuration unit can add candidates generated by averaging pairs of predefined candidates in the merged candidate list to the merged candidate list.

[0026] Beneficial effects

[0027] According to embodiments of this disclosure, various highly reliable merging candidates are additionally used to enhance the prediction accuracy and compression performance of the merging pattern.

[0028] Furthermore, according to embodiments of this disclosure, motion vectors of merged candidates are corrected by motion vector offset to increase the accuracy of motion estimation / compensation.

[0029] The effects that can be obtained in this disclosure are not limited to those mentioned above, and those skilled in the art will clearly understand other unmentioned effects based on the following description. Attached Figure Description

[0030] To aid in understanding this disclosure, the accompanying drawings, which are included as part of the detailed description, provide embodiments of the disclosure and together with the detailed description, describe the technical features of the disclosure.

[0031] Figure 1 This is a schematic block diagram of an encoding device for performing video / image signal encoding as an embodiment of the present disclosure.

[0032] Figure 2 This is a schematic block diagram of a decoding device that performs video / image signal decoding as an embodiment of the present disclosure.

[0033] Figure 3 This is a diagram illustrating examples of various types of tree structures that can be applied to this disclosure.

[0034] Figure 4 This is a diagram illustrating a signaling mechanism for segmentation information of a quadtree with nested multi-type tree structures, which can be implemented using the present disclosure.

[0035] Figure 5 This diagram illustrates a method for dividing a CTU into multiple CUs based on quadtrees and nested multi-type tree structures, which can be implemented using the present disclosure.

[0036] Figure 6 This is an illustration of a method for limiting ternary tree segmentation as an embodiment of which the present disclosure can be applied.

[0037] Figure 7 This is a diagram illustrating a redundant partitioning pattern that may occur during binary tree partitioning and ternary tree partitioning, as an embodiment of which this disclosure can be applied.

[0038] Figure 8 and Figure 9 This is an illustration of a video / image coding method based on inter-frame prediction according to an embodiment of the present disclosure and an inter-frame prediction unit in a coding apparatus according to an embodiment of the present disclosure.

[0039] Figure 10 and Figure 11 The diagram illustrates a video / image decoding method based on inter-frame prediction according to an embodiment of the present disclosure and an inter-frame prediction unit in a decoding apparatus according to an embodiment of the present disclosure.

[0040] Figure 12This is a diagram used to describe adjacent blocks used in merge mode or skip mode as an implementation of this disclosure.

[0041] Figure 13 This is a flowchart illustrating a method for configuring and merging candidate lists according to an embodiment of the present disclosure.

[0042] Figure 14 This is a flowchart illustrating a method for configuring and merging candidate lists according to an embodiment of the present disclosure.

[0043] Figure 15 This is a diagram illustrating a method for deriving advanced time motion vector prediction (ATMVP) candidates as an implementation of this disclosure.

[0044] Figure 16 This is a diagram illustrating a method for deriving advanced time motion vector prediction (ATMVP) candidates as an implementation of this disclosure.

[0045] Figure 17 and Figure 18 This is a diagram illustrating a method for compressing time motion vector data and the location of spatial candidates therefor, according to an embodiment of the present disclosure.

[0046] Figure 19 This diagram illustrates a method for configuring a merging candidate list according to conventional image compression techniques, which can be applied as an embodiment of this disclosure.

[0047] Figure 20 This is a flowchart illustrating a method for generating modified merge candidates as an embodiment of the present disclosure.

[0048] Figure 21 This is an illustration of a motion vector offset for generating corrected merging candidates as an embodiment of the present disclosure.

[0049] Figure 22 This is a flowchart illustrating a method for generating inter-frame prediction blocks according to an embodiment of the present disclosure.

[0050] Figure 23 This is a diagram illustrating an inter-frame prediction apparatus according to an embodiment of the present disclosure.

[0051] Figure 24 An example of a video coding system using this disclosure is provided.

[0052] Figure 25 This is an architectural diagram of a content streaming system as an embodiment of the present disclosure. Detailed Implementation

[0053] Some embodiments of this disclosure will be described in more detail with reference to the accompanying drawings. Figure 1 The detailed description disclosed herein is intended to describe some embodiments of this disclosure, and not the only embodiments thereof. The following detailed description includes further details to provide a complete understanding of this disclosure. However, those skilled in the art should understand that this disclosure can be implemented without these details.

[0054] In some cases, to avoid obscuring the concepts of this disclosure, known structures and devices are omitted, or may be shown in block diagram form based on the core functions of each structure and device.

[0055] While most of the terms used in this disclosure are selected from commonly used terms in the art, some terms have been arbitrarily chosen by the applicant, and their meanings will be explained in detail in the following description as needed. Therefore, this disclosure should be understood based on the literal meaning of the terms rather than their simple names or connotations.

[0056] The specific terminology used in the following description is provided to aid in understanding this disclosure, and the use of such specific terminology may be modified in various ways without departing from the technical spirit of this disclosure. For example, signals, data, samples, images, frames, blocks, etc., may be appropriately replaced and interpreted in each encoding process.

[0057] In this specification, a "processing unit" refers to a unit in which encoding / decoding processes such as prediction, transformation, and / or quantization are performed. Hereinafter, for ease of description, a processing unit may be referred to as a "processing block" or a "block".

[0058] Furthermore, the term "processing unit" can be interpreted to include units for the luminance component and units for the chrominance component. For example, a processing unit may correspond to a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

[0059] Furthermore, a processing unit can be interpreted as a unit for the luma component or a unit for the chroma component. For example, a processing unit can correspond to a coding tree block (CTB), coding block (CB), prediction unit (PU), or transform block (TB) for the luma component. Additionally, a processing unit can correspond to a CTB, CB, PU, ​​or TB for the chroma component. Moreover, the processing unit is not limited to these definitions and can be interpreted to include both units for the luma component and units for the chroma component.

[0060] In addition, the processing unit is not necessarily limited to a square block, and can be configured as a polygon shape with three or more vertices.

[0061] Furthermore, in this specification, pixels are referred to as samples. Additionally, using samples can mean using pixel values, etc.

[0062] Figure 1 This is a schematic block diagram of an encoding device for encoding video / image signals as an embodiment of the present disclosure.

[0063] Reference Figure 1 The encoding device 100 can be configured to include an image divider 110, a subtractor 115, a transformer 120, a quantizer 130, an inverse quantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-frame predictor 180, an intra-frame predictor 185, and an entropy encoder 190. The inter-frame predictor 180 and the intra-frame predictor 185 can be collectively referred to as predictors. In other words, predictors can include inter-frame predictor 180 and intra-frame predictor 185. The transformer 120, quantizer 130, inverse quantizer 140, and inverse transformer 150 can be included in a residual processor. The residual processor can also include subtractor 115. In one implementation, the image segmenter 110, subtractor 115, transformer 120, quantizer 130, dequantizer 140, inverse transformer 150, adder 155, filter 160, inter-frame predictor 180, intra-frame predictor 185, and entropy encoder 190 can be configured as a single hardware component (e.g., an encoder or processor). Furthermore, the memory 170 may include a decoded image buffer (DPB) and may be implemented using a digital storage medium.

[0064] Image segmenter 110 divides the input image (or picture or frame) input to encoding device 100 into one or more processing units. For example, a processing unit may be referred to as a coding unit (CU). In this case, coding units can be recursively segmented from coding tree units (CTUs) or maximum coding units (LCUs) based on a quadtree-binary tree (QTBT) structure. For example, a coding unit can be segmented into multiple deeper coding units based on a quadtree structure and / or a binary tree structure. In this case, for example, a quadtree structure can be applied first, followed by a binary tree structure. Alternatively, a binary tree structure can be applied first. The encoding process according to this disclosure can be performed based on the final coding units that are no longer segmented. In this case, the maximum coding unit can be directly used as the final coding unit based on encoding efficiency according to image characteristics, or, if necessary, the coding unit can be recursively segmented into deeper coding units. Thus, a coding unit with an optimal size can be used as the final coding unit. In this case, the encoding process may include processes such as prediction, transformation, or reconstruction, which will be described subsequently. In another example, the processing unit may also include a prediction unit (PU) or a transform unit (TU). In this case, each of the prediction unit and the transform unit can be partitioned or segmented from each final coding unit. The prediction unit may be a unit that predicts samples, and the transform unit may be a unit that derives transform coefficients from it and / or derives the residual signal from the transform coefficients.

[0065] Units can be used interchangeably with blocks or regions, depending on the situation. In common cases, an M×N block can refer to a set of samples or a set of transform coefficients consisting of M columns and N rows. Typically, a sample can indicate a pixel or pixel value, and can indicate pixel / pixel values ​​for only the luminance component or only the chrominance component. Within a sample, an image (or picture) can be used as the term corresponding to a pixel or graphic element.

[0066] Encoding device 100 can generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output by inter-frame predictor 180 or intra-frame predictor 185 from the input image signal (original block or original sample array). The generated residual signal is sent to converter 120. In this case, as illustrated, the unit that subtracts the prediction signal (prediction block or prediction sample array) from the input image signal (original block or original sample array) within encoding device 100 can be referred to as subtractor 115. The predictor can perform prediction on the processing target block (hereinafter referred to as the current block) and can generate a prediction block that includes the prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction on a unit of the current block or CU. The predictor can generate various information about the prediction, such as prediction mode information, as will be described later in the description of each prediction mode, and can send the information to entropy encoder 190. The information about the prediction can be encoded in entropy encoder 190 and can be output in the form of a bitstream.

[0067] The intra-frame predictor 185 can refer to samples within the current image to predict the current block. The referenced samples can be located as neighbors of the current block, or spaced apart from the current block depending on the prediction mode. In intra-frame prediction, the prediction mode can include multiple non-angular modes and multiple angular modes. For example, non-angular modes can include DC mode and planar mode. For example, depending on the fineness of the prediction direction, angular modes can include 33 or 65 angular prediction modes. In this case, for example, depending on the configuration, more or fewer than 33 or 65 angular prediction modes can be used. The intra-frame predictor 185 can use the prediction modes applied to neighboring blocks to determine the prediction mode applied to the current block.

[0068] Inter-frame predictor 180 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by motion vectors on a reference image. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information of neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks within the current image and temporally neighboring blocks within the reference image. The reference image including the reference block and the reference image including the temporally neighboring block may be the same or different. The temporally neighboring block may be represented by a name called a co-located reference block or co-located CU (colCU). The reference image including the temporally neighboring block may be called a co-located image (colPic). For example, inter-frame predictor 180 can construct a motion information candidate list based on neighboring blocks and can generate information indicating which candidate is used to deduce the motion vector and / or reference image index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the inter-frame predictor 180 can use motion information from neighboring blocks as motion information for the current block. In skip mode, unlike merge mode, residual signals cannot be sent. In motion vector prediction (MVP) mode, motion vectors from neighboring blocks can be used as motion vector predictors. The motion vector of the current block can be indicated by signaling the motion vector difference.

[0069] The predicted signal generated by the inter-frame predictor 180 or the intra-frame predictor 185 can be used to generate the reconstructed signal or the residual signal.

[0070] Transformer 120 can generate transform coefficients by applying a transform scheme to the residual signal. For example, the transform scheme may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT). In this case, GBT refers to a transform obtained from a graph when representing the relationship information between pixels as a graph. CNT refers to a transform obtained based on a predicted signal generated using all previously reconstructed pixels. Furthermore, the transform process can be applied to square blocks of pixels of the same size, or it can be applied to blocks of variable size that are not square.

[0071] Quantizer 130 quantizes the transform coefficients and transmits them to entropy encoder 190. Entropy encoder 190 encodes the quantized signal (information about the quantized transform coefficients) and outputs it as a bitstream. This information about the quantized transform coefficients can be referred to as residual information. Quantizer 130 can rearrange the block-form quantized transform coefficients into a one-dimensional vector form based on the coefficient scan order, and can generate information about the quantized transform coefficients based on this one-dimensional vector form. Entropy encoder 190 can perform various encoding methods such as exponential Golomb, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). In addition to the quantized transform coefficients, entropy encoder 190 can also encode information necessary for video / image reconstruction (e.g., values ​​of syntax elements) together or separately. The encoded information (e.g., encoded video / image information) can be transmitted or stored as a bitstream in units of Network Abstraction Layer (NAL). The bitstream can be transmitted over a network or stored in a digital storage medium. In this configuration, the network may include a broadcast network and / or a communication network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmitter (not shown) that sends the signal output by the entropy encoder 190 and / or a memory (not shown) that stores the signal may be configured as an internal / external element of the encoding device 100, or the transmitter may be an element of the entropy encoder 190.

[0072] The quantized transform coefficients output by quantizer 130 can be used to generate a prediction signal. For example, the residual signal can be reconstructed by applying inverse quantization and inverse transform to the quantized transform coefficients using inverse quantizer 140 and inverse transformer 150 within the loop. Adder 155 can add the reconstructed residual signal to the prediction signal output by inter-frame predictor 180 or intra-frame predictor 185, thus generating a reconstructed signal (reconstructed image, reconstructed block, or reconstructed sample array). If the target block has no residual, as in the case where skip mode has been applied, the prediction block can be used as a reconstructed block. Adder 155 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block within the current image, and, as described later, for inter-frame prediction of the next image after filtering.

[0073] Filter 160 can improve the quality of the subjective / objective image by applying filtering to the reconstructed signal. For example, filter 160 can generate a corrected reconstructed image by applying various filtering methods to the reconstructed image. The corrected reconstructed image can be stored in memory 170, more specifically, in the DPB of memory 170. Various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, and bilateral filtering. Filter 160 can generate information for filtering as described below in the description of each filtering method, and can send it to entropy encoder 190. The filtering information can be encoded by entropy encoder 190 and output as a bitstream.

[0074] The corrected reconstructed image sent to memory 170 can be used as a reference image in inter-frame predictor 180. If inter-frame prediction is applied, the encoding device can avoid prediction mismatch between encoding device 100 and decoding device and improve encoding efficiency.

[0075] The DPB of memory 170 can store the corrected reconstructed image for use as a reference image in inter-frame predictor 180. Memory 170 can store motion information of blocks in which motion information in the current image is derived (or encoded) and / or motion information of blocks in already reconstructed images. The stored motion information can be forwarded to inter-frame predictor 180 to be used as motion information for spatially or temporally neighboring blocks. Memory 170 can store reconstructed samples of reconstructed blocks in the current image and forward them to intra-frame predictor 185.

[0076] Figure 2 This is an embodiment of the present disclosure and a schematic block diagram of a decoding device for decoding video / image signals.

[0077] Reference Figure 2 The decoding device 200 can be configured to include an entropy decoder 210, an inverse quantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame predictor 260, and an intra-frame predictor 265. The inter-frame predictor 260 and the intra-frame predictor 265 can be collectively referred to as predictors. That is, predictors can include inter-frame predictor 180 and intra-frame predictor 185. The inverse quantizer 220 and the inverse transformer 230 can be collectively referred to as a residual processor. That is, a residual processor can include inverse quantizer 220 and inverse transformer 230. The entropy decoder 210, inverse quantizer 220, inverse transformer 230, adder 235, filter 240, inter-frame predictor 260, and intra-frame predictor 265 can be configured as a single hardware component (e.g., a decoder or a processor) depending on the implementation. Furthermore, the memory 250 can include a decoded picture buffer (DPB) and can be implemented using a digital storage medium.

[0078] When the input includes a bitstream containing video / image information, the decoding device 200 can proceed according to... Figure 1 The image is reconstructed through processing of video / image information in the encoding device. For example, the decoding device 200 can perform decoding using a processing unit applied in the encoding device. Therefore, the processing unit used for decoding can be, for example, an encoding unit. Depending on the quadtree structure and / or binary tree structure, encoding units can be segmented from encoding tree units or maximal encoding units. Furthermore, the reconstructed image signal decoded and output by the decoding device 200 can be played back by a playback device.

[0079] Decoding device 200 can receive data in bitstream form from... Figure 1 The signal output by the encoding device. The received signal can be decoded by the entropy decoder 210. For example, the entropy decoder 210 can deduce information (e.g., video / image information) for image reconstruction (or picture reconstruction) by parsing the bitstream. For example, the entropy decoder 210 can decode the information within the bitstream based on encoding methods such as exponential Golomb coding, CAVLC, or CABAC, and can output the values ​​of syntax elements for image reconstruction or quantized values ​​of transform coefficients with respect to the residuals. More specifically, in the CABAC entropy decoding method, bins corresponding to each syntax element can be received from the bitstream, and a context model can be determined using the decoding information of neighboring and target blocks, as well as the decoding target syntax element information or information of symbols / bins decoded in previous steps. The probability of bin occurrence can be predicted based on the determined context model, and symbols corresponding to the values ​​of each syntax element can be generated by performing arithmetic decoding on the bins. In this case, in the CABAC entropy decoding method, after determining the context model, the context model can be updated using information of symbols / bins decoded for the context model of the next symbol / bin. Information about prediction from the information decoded in entropy decoder 2110 can be provided to the predictors (inter-frame predictor 260 and intra-frame predictor 265). Parameter information (i.e., quantized transform coefficients) related to the residual values ​​from which entropy decoding has been performed in entropy decoder 210 can be input to dequantizer 220. Furthermore, information about filtering from the information decoded in entropy decoder 210 can be provided to filter 240. Additionally, a receiver (not shown) that receives the signal output from the encoding device can be configured as an internal / external element of decoding device 200, or the receiver can be a component of entropy decoder 210.

[0080] The dequantizer 220 can dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 can rearrange the quantized transform coefficients into two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order executed in the encoding device. The dequantizer 220 can perform dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain the transform coefficients.

[0081] The inverse converter 230 can output the residual signal (residual block or residual sample array) by applying an inverse transform to the transform coefficients.

[0082] The predictor can perform predictions on the current block and generate a prediction block that includes prediction samples for the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the prediction information output by the entropy decoder 210, and can determine the detailed intra-frame / inter-frame prediction mode.

[0083] The intra-predictor 265 can refer to samples within the current image to predict the current block. Depending on the prediction mode, the referenced samples can be located as neighbors of the current block or as spaced apart from the current block. In intra-prediction, the prediction mode can include multiple non-angular modes and multiple angular modes. The intra-predictor 265 can use the prediction modes applied to neighboring blocks to determine the prediction mode applied to the current block.

[0084] Inter-frame predictor 260 can deduce the predicted block for the current block based on a reference block (reference sample array) specified by motion vectors on a reference image. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information of neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks within the current image and temporally neighboring blocks within the reference image. For example, inter-frame predictor 260 can configure a motion information candidate list based on neighboring blocks and can deduce the motion vector and / or reference image index for the current block based on received candidate selection information. Inter-frame prediction can be performed based on various prediction modes. Information about the prediction may include information indicating the mode of inter-frame prediction for the current block.

[0085] Adder 235 can generate a reconstruction signal (reconstructed image, reconstruction block, or reconstruction sample array) by adding the obtained residual signal to the prediction signal (prediction block or prediction sample array) output by inter-frame predictor 260 or intra-frame predictor 265. If the target block has no residual, as in the case where skip mode has been applied, the prediction block can be used as the reconstruction block.

[0086] Adder 235 can be referred to as a reconstructor or reconstruction block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block within the current image, and, as described later, for inter-frame prediction of the next image after filtering.

[0087] Filter 240 can improve the quality of subjective / objective images by applying filtering to the reconstructed signal. For example, filter 240 can generate a corrected reconstructed image by applying various filtering methods to the reconstructed image, and the corrected reconstructed image can be sent to memory 250, more specifically, to the DPB of memory 250. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset (SAO), adaptive loop filter (ALF), and bilateral filter.

[0088] The (corrected) reconstructed image stored in the DPB of memory 250 can be used as a reference image in inter-frame predictor 260. Memory 250 can store motion information of blocks in which motion information in the current image is derived (or decoded) and / or motion information of blocks in already reconstructed images. The stored motion information can be forwarded to inter-frame predictor 260 to be used as motion information for spatially or temporally neighboring blocks. Memory 170 can store reconstructed samples of reconstructed blocks in the current image and forward them to intra-frame predictor 265.

[0089] In this disclosure, the embodiments described in the filter 160, inter-frame predictor 180 and intra-frame predictor 185 of the encoding device 100 can be applied to the filter 240, inter-frame predictor 260 and intra-frame predictor 265 of the decoding device 200 in the same or corresponding manner.

[0090] Block segmentation

[0091] The video / image coding method according to this disclosure can be performed based on various detailed techniques, each of which is described below. It will be apparent to those skilled in the art that the techniques described herein can be associated with related processes such as prediction, residual processing (inverse transform, inverse quantization, etc.), syntax element encoding, filtering, and segmentation / partitioning in the video / image coding / decoding process, as described above and / or below.

[0092] The block segmentation process according to this disclosure can be performed in the image segmenter 110 of the aforementioned encoding device, and the segmentation-related information can be (encoded) in the entropy encoder 190 and forwarded to the decoding device in bitstream format. The entropy decoder 210 of the decoding device can obtain the block segmentation structure of the current image based on the segmentation-related information obtained from the bitstream, and based on this, a series of processes (e.g., prediction, residual processing, block reconstruction, and in-loop filtering, etc.) can be performed for image decoding.

[0093] Segment the image into CTUs

[0094] An image can be divided into a series of coding tree units (CTUs). Each CTU can correspond to a coding tree block (CTB). Alternatively, a CTU may include a coding tree block for luma samples and two corresponding coding tree blocks for chroma samples. In other words, for an image containing arrays of three types of samples, a CTU may include an N×N block of luma samples and two corresponding samples for chroma samples.

[0095] The maximum support size of the CTU used for encoding and prediction can differ from the maximum support size of the CTU used for transformation. For example, the maximum support size of the luma block in the CTU can be 128×128.

[0096] CTU segmentation using a tree structure

[0097] The CTU can be partitioned into CUs based on a quadtree (QT) structure. The quadtree structure can be referred to as a quaternion structure. This is to reflect various local characteristics. Furthermore, in this disclosure, the CTU can be partitioned based on multiple types of tree structures, including binary trees (BT), ternary trees (TT), and quadtrees. In the following, the QTBT structure can include quadtree and binary tree structures, and QTBTTT can include partitioned structures based on binary and ternary trees. Alternatively, the QTBT structure can also include partitioned structures based on quadtrees, binary trees, and ternary trees. In the coding tree structure, the CU can have a square or rectangular shape. The CTU can first be partitioned into a quadtree structure. Then, the leaf nodes of the quadtree structure can be further partitioned according to multiple types of tree structures.

[0098] Figure 3 This is a diagram illustrating an example of a multi-type tree structure to which embodiments of the present disclosure can be applied.

[0099] In embodiments of this disclosure, the multi-type tree structure may include four partitioning types, such as Figure 3As shown in the diagram, the four segmentation types can include vertical binary segmentation (SPLIT_BT_VER), horizontal binary segmentation (SPLIT_BT_HOR), vertical ternary segmentation (SPLIT_TT_VER), and horizontal ternary segmentation (SPLIT_TT_HOR). Leaf nodes of the multi-type tree structure can be referred to as CUs. Such CUs can be used in prediction and transformation processes. In this disclosure, typically, CUs, PUs, and TUs can have the same block size. However, CUs and TUs can have different block sizes when the maximum supported transformation length is less than the width or height of the color component.

[0100] Figure 4 This is a diagram illustrating a signaling mechanism for partitioning information of a quadtree with nested multi-type tree structures, which is an embodiment of the present disclosure that can be applied.

[0101] Here, the CTU can be considered the root of a quadtree and is initially split into a quadtree structure. Each quadtree leaf node can then be further split into a multi-type tree structure. In the multi-type tree structure, a first flag (e.g., `mtt_split_cu_flag`) is signaled to indicate whether the corresponding node should be further split. If the corresponding node is further split, a second flag (e.g., `mtt_split_cu_vertical_flag`) can be signaled to indicate the split direction. Subsequently, a third flag (e.g., `mtt_split_cu_binary_flag`) can be signaled to indicate whether the split type is a binary split or a ternary split. For example, based on `mtt_split_cu_vertical_flag` and `mtt_split_cu_binary_flag`, the multi-type tree splitting mode (MttSplitMode) can be derived as shown in Table 1 below.

[0102] [Table 1]

[0103] MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1

[0104] Figure 5 This diagram illustrates a method for dividing a CTU into multiple CUs based on quadtrees and nested multi-type tree structures, which can be implemented using the present disclosure.

[0105] Here, bolded edges indicate quadtree partitions, while the remaining edges indicate multi-type tree partitions. Quadtree partitions with nested multi-type trees can provide a content-adaptive coding tree structure. A CTU can correspond to a coding block (CB). Alternatively, a CU can include a coding block for luma samples and two coding blocks for corresponding chroma samples. The size of a CU can be as small as 4×4, or as large as a CTU, measured in luma samples. For example, in a 4:2:0 color format (or chroma format), the maximum chroma CB size can be 64×64, and the minimum chroma CB size can be 2×2.

[0106] In this disclosure, for example, the maximum supported luminance TB size can be 64×64, and the maximum supported chrominance TB size can be 32×32. If the width or height of a CB segmented according to the tree structure is greater than the maximum transform width or height, the CB can be further segmented until the TB size limits in the horizontal and vertical directions are automatically (or implicitly) satisfied.

[0107] Furthermore, for quadtree encoding tree schemes with nested multi-type trees, the following parameters can be defined or identified as SPS syntax elements.

[0108] -CTU size: Size of the root node of the quadtree

[0109] -MinQTSize: Minimum allowed size of quadtree leaf nodes

[0110] -MaxBtSize: Maximum allowed size of the root node of the binary tree

[0111] -MaxTtSize: Maximum allowed size of the root node of the ternary tree

[0112] -MaxMttDepth: The maximum allowed depth of hierarchical division of a multi-type tree from quadtree leaves.

[0113] -MinBtSize: Minimum allowed size of binary leaf nodes

[0114] -MinTtSize: Minimum allowed size of a ternary leaf node

[0115] As an example of a quadtree coding tree scheme with nested multi-type trees, the CTU size (in 4:2:0 chroma format) can be set to 128×128 luma samples and two corresponding chroma samples in 64×64 blocks. In this case, MinOTSize can be set to 16×16, MaxBtSize can be set to 128×128, MaxTtSize can be set to 64×64, MinBtSize and MinTtSize (for both width and height) can be set to 4×4, and MaxMttDepth can be set to 4. Quadtree partitioning can be applied to the CTU and generate quadtree leaf nodes. Quadtree leaf nodes can be called leaf QT nodes. Quadtree leaf nodes can have sizes ranging from 16×16 (i.e., MinOTSize) to 128×128 (i.e., CTU size). In the case of a 128×128 leaf QT node, the leaf QT node cannot be partitioned into a binary / ternary tree. This is because even when leaf QT nodes are split, they exceed MaxBtsize and MaxTtsize (i.e., 64×64). In other cases, leaf QT nodes can be further split into multi-type trees. Therefore, a leaf QT node can be the root node for a multi-type tree, and it can have a multi-type tree depth (mttDepth) of 0. When the multi-type tree depth reaches MaxMttdepth (e.g., 4), additional splits are no longer considered. When the width of a multi-type tree node is equal to MinBtSize and less than or equal to 2×MinTtSize, additional horizontal splits are no longer considered. When the height of a multi-type tree node is equal to MinBtSize and less than or equal to 2×MinTtSize, additional vertical splits are no longer considered.

[0116] Figure 6 This is an illustration of a method for limiting ternary tree segmentation as an embodiment of which the present disclosure can be applied.

[0117] Reference Figure 6 To support the 64×64 luma block and 32×32 chroma pipeline design in the hardware decoder, TT segmentation can be limited under certain circumstances. For example, in situations such as... Figure 6 If the width or height of the luminance coding block shown is greater than a predetermined specific value (e.g., 32, 64), TT segmentation can be restricted.

[0118] In this disclosure, the coding tree scheme can support luma and chroma blocks having separate block tree structures. Relative to P-slices and B-slices, the luma and chroma CTBs within a single CTU can be restricted to having the same coding tree structure. However, for I-slices, luma and chroma blocks can each have their own separate block tree structures. With each block tree pattern applied, the luma CTB can be segmented into CUs based on a specific coding tree structure, and the chroma CTB can be segmented into chroma CUs based on different coding tree structures. This can mean that a CU in an I-slice can include a coding block for the luma component or a coding block for two chroma components, and a CU in a P- or B-slice can include a block for three color components.

[0119] In the above "Splitting CTU using a Tree Structure," a quadtree encoding scheme with nested multi-type trees was described, but the splitting structure of the CU is not limited to this. For example, the BT structure and TT structure can be interpreted as concepts included in the Multi-Split Tree (MPT) structure, and can be interpreted as the CU being split by the QT structure and the MPT structure. In the example of splitting the CU by the QT structure and the MPT structure, syntax elements (e.g., MPT_split_type) including information about the number of blocks into which the leaf nodes of the QT structure are split, and syntax elements (e.g., MPT_split_mode) including information about the direction in which the leaf nodes of the QT structure are split, in the vertical and horizontal directions, can be signaled, and the splitting structure can be determined.

[0120] In another example, the CU can be partitioned in a way different from the QT, BT, or TT structures. That is, unlike the lower-depth CU being partitioned into 1 / 4 the size of the higher-depth CU according to the QT structure, the lower-depth CU being partitioned into 1 / 2 the size of the higher-depth CU according to the BT structure, or the lower-depth CU being partitioned into 1 / 4 or 1 / 2 the size of the higher-depth CU according to the TT structure, in some cases the lower-depth CU can be partitioned into 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 the size of the higher-depth CU, but the method of partitioning the CU is not limited to these.

[0121] If a portion of a tree node block extends beyond the bottom or right image boundary, the corresponding tree node block can be restricted so that all samples of all encoded CUs are within the image boundary. In this case, for example, the following segmentation rule can be applied.

[0122] -If part of a tree node block extends beyond both the bottom and right image boundaries.

[0123] - If the block is a QT node and the block size is greater than the minimum QT size, then the block is forced to be split in QT splitting mode.

[0124] Otherwise, force the blocks to be split in the Split_BT_HOR mode.

[0125] Otherwise, if part of a tree node block extends beyond the bottom image boundary,

[0126] - If the block is a QT node and the block size is greater than the minimum QT size and the block size is greater than the maximum BT size, then the block is forced to be split in QT splitting mode.

[0127] Otherwise, if the block is a QT node and the block size is greater than the minimum QT size and the block size is less than or equal to the maximum BT size, then the block is forced to be split in either QT split mode or SPLIT_BT_HOR mode.

[0128] Otherwise (if the block is a BTT node or the block size is less than or equal to the minimum QT size), the block will be forced to be split in the SPLIT_BT_HOR mode.

[0129] Otherwise, if part of the tree node block extends beyond the right image boundary,

[0130] - If the block is a QT node and the block size is greater than the minimum QT size and the block size is greater than the maximum BT size, then the block is forced to be split in QT splitting mode.

[0131] Otherwise, if the block is a QT node and the block size is greater than the minimum QT size and the block size is less than or equal to the maximum BT size, then the block is forced to be split in QT split mode or SPLIT_BT_VER mode.

[0132] Otherwise (if the block is a BTT node or the block size is less than or equal to the minimum QT size), the block will be forced to be split in SPLIT_BT_VER mode.

[0133] The quadtree coding block structure accompanying multi-type trees can provide a very flexible block partitioning structure. Due to the various partitioning types supported by multi-type trees, different partitioning patterns may result in the same coding block structure in certain situations. The generation of redundant partitioning patterns is limited to reduce the amount of data involved in the partitioning information. This will be described with reference to the following figure.

[0134] Figure 7 This is a diagram illustrating a redundant partitioning pattern that may occur during binary tree partitioning and ternary tree partitioning, as an embodiment of which this disclosure can be applied.

[0135] like Figure 7As illustrated, a binary segmentation with two consecutive levels in one direction has the same coded block structure as the binary segmentation used for center segmentation after a ternary segmentation. In this case, the binary segmentation (in a given direction) used for center segmentation in ternary segmentation can be restricted. This restriction can be applied to the CUs of all images. When a particular segmentation is restricted, the signaling of the syntax elements can be modified to reflect this restriction, and the number of bits signaled for segmentation can be reduced through the modified signaling. For example, as... Figure 7 In the example shown, when the binary tree split used for the center split of the CU is restricted, the syntax element mtt_split_cu_binary_flag indicating whether the split is a binary split or a ternary split can be left unsigned, and this value can be inferred to be 0 by the decoder.

[0136] predict

[0137] To reconstruct the current processing unit in which decoding is performed, the decoded portion of the current image or other images including the current processing unit can be used.

[0138] An image that is reconstructed using only the current image (i.e., performs intra-frame prediction) can be called an intra-frame image or I-image (slice). An image (slice) that uses up to one motion vector and reference index to predict each cell can be called a prediction image or P-image (slice). An image (slice) that uses up to two motion vectors and reference indices can be called a bidirectional prediction image or B-image (slice).

[0139] Intra-frame prediction refers to a method of deriving predictions for the current processing block from data elements (e.g., sample values) of the same decoded image (or slice). In other words, intra-frame prediction means predicting the pixel values ​​of the current processing block by referring to the reconstructed regions in the current image.

[0140] Inter-frame prediction will be described in more detail below.

[0141] Inter-frame prediction

[0142] Inter-frame prediction refers to a method of deriving predictions for the current processing block based on data elements (e.g., sample values ​​or motion vectors) from images other than the current image. In other words, inter-frame prediction means predicting the pixel values ​​of the current processing block by referring to reconstructed regions in other reconstructed images besides the current image.

[0143] Inter-frame prediction (inter-picture prediction), a technique for eliminating redundancy between images, is primarily performed through motion estimation and motion compensation.

[0144] In this disclosure, the above Figure 1 and Figure 2 The inter-frame prediction method described in the document is described in detail, and the decoder can be represented as described below. Figure 10 Video / image decoding methods based on inter-frame prediction and Figure 11 The inter-frame prediction unit in the decoding device. Furthermore, the encoder can be represented as described below. Figure 8 Video / image coding methods based on inter-frame prediction and Figure 9 The inter-frame prediction unit in the coding device. Additionally, through... Figure 8 and Figure 9 Encoded data can be stored as a bit stream.

[0145] The prediction unit of an encoding / decoding device can derive prediction samples by performing inter-frame prediction on a block-by-block basis. Inter-frame prediction can be represented as a prediction derived in a way that depends on data elements (e.g., sample values ​​or motion information) of images other than the current image. When applying inter-frame prediction to the current block, the prediction block (prediction sample array) of the current block can be derived based on the reference block (reference sample array) specified by the motion vector on the reference image indicated by the reference image index.

[0146] In this scenario, to reduce the amount of motion information transmitted in inter-frame prediction mode, the motion information of the current block can be predicted on a block, sub-block, or sample basis based on the correlation between the motion information of neighboring blocks and the current block. Motion information can include motion vectors and reference image indices. It can also include inter-frame prediction type information (L0 prediction, L1 prediction, Bi prediction, etc.).

[0147] When applying inter-frame prediction, neighboring blocks can include spatially neighboring blocks present in the current image and temporally neighboring blocks present in a reference image. The reference image including the reference block and the reference image including the temporally neighboring block can be the same as or different from each other. Temporally neighboring blocks can be referred to by names such as juxtaposed reference blocks, juxtaposed CUs (colCU), etc., and the reference image including the temporally neighboring block can be referred to as a juxtaposed image (colPic). For example, a motion information candidate list can be configured based on the neighboring blocks of the current block, and to derive the motion vector and / or reference image index of the current block, a signal can be sent indicating which candidate's flag or index information to select (use).

[0148] Inter-frame prediction can be performed based on various prediction modes, and for example, in skip mode and merge mode, the motion information of the current block can be the same as the motion information of the selected neighboring blocks. In the case of skip mode, residual signals can be sent instead of in merge mode. In the case of motion vector prediction (MVP) mode, the motion vectors of the selected neighboring blocks can be used as motion vector predictors, and the motion vector difference can be signaled. In this case, the motion vector of the current block can be derived by using the sum of the motion vector predictors and the motion vector difference.

[0149] Figure 8 and Figure 9 This is an illustration of a video / image coding method based on inter-frame prediction according to an embodiment of the present disclosure and an inter-frame prediction unit in a coding apparatus according to an embodiment of the present disclosure.

[0150] Reference Figure 8 and Figure 9 S801 can be executed by the inter-frame prediction unit 180 of the encoding device, and S802 can be executed by the residual processing unit of the encoding device. Specifically, S802 can be executed by the subtractor 115 of the encoding device. In S803, the prediction information can be derived by the inter-frame prediction unit 180 and encoded by the entropy encoder 190. In S803, the residual information can be derived by the residual processing unit and encoded by the entropy encoder 190. The residual information is information about the residual samples. The residual information may include information about the quantized transform coefficients for the residual samples.

[0151] As described above, the residual samples can be derived into transform coefficients by the transformer 120 of the encoding device, and the transform coefficients can be derived into transform coefficients quantized by the quantizer 130. The information about the quantized transform coefficients can be encoded by the entropy encoder 190 using the residual encoding process.

[0152] The encoding device performs inter-frame prediction on the current block (S801). The encoding device can deduce the inter-frame prediction mode and motion information of the current block, and generate prediction samples for the current block. Here, the inter-frame prediction mode determination process, the motion information derivation process, and the prediction sample generation process can be executed simultaneously, and any one of these processes can be executed earlier than the other processes. For example, the inter-frame prediction unit 180 of the encoding device may include a prediction mode determination unit 181, a motion information derivation unit 182, and a prediction sample derivation unit 183. The prediction mode determination unit 181 can determine the prediction mode of the current block, the motion information derivation unit 182 can deduce the motion information of the current block, and the prediction sample derivation unit 183 can deduce the motion samples of the current block.

[0153] For example, the inter-frame prediction unit 180 of the encoding apparatus can search for blocks similar to the current block in a predetermined region (search region) of a reference image through motion estimation, and deduce a reference block whose difference from the current block is minimal, equal to, or less than a predetermined standard. Based on this, a reference image index indicating the location of the reference block can be derived, and motion vectors can be derived based on the positional difference between the reference block and the current block. The encoding apparatus can determine the prediction mode applicable to the current block. The encoding apparatus can compare the RD costs of various prediction modes and determine the optimal prediction mode for the current block.

[0154] For example, when applying a skip mode or merge mode to the current block, the encoding device can configure a merge candidate list, as described below, and deduce a reference block among the reference blocks indicated by the merge candidates included in the merge candidate list whose difference from the current block is minimal, equal to, or less than a predetermined standard. In this case, a merge candidate associated with the deduced reference block can be selected, and merge index information indicating the selected merge candidate can be generated and signaled to the decoding device. The motion information of the current block can be deduced using the motion information of the selected merge candidate.

[0155] As another example, when applying the (A)MVP mode to the current block, the decoding device can configure an (A)MVP candidate list as described below, and use the motion vector of the MVP candidate selected from the motion vector prediction sub-MVP candidates included in the (A)MVP candidate list as the MVP of the current block. In this case, for example, the motion vector of a reference block derived through motion estimation can be used as the motion vector of the current block, and the MVP candidate with the smallest difference from the motion vector of the current block can be selected as the MVP candidate. The motion vector difference (MVD) can be derived as the difference obtained by subtracting the MVP from the motion vector of the current block. In this case, information about the MVD can be signaled to the decoding device. Additionally, when applying the (A)MVP mode, the value of the reference image index can be configured as reference image index information and signaled separately to the decoding device.

[0156] The encoding device can derive residual samples based on the predicted samples (S802). The encoding device can derive residual samples by comparing the original samples of the current block with the predicted samples.

[0157] The encoding device encodes image information including prediction information and residual information (S803). The encoding device can output the encoded image information in the form of a bitstream. The prediction information may include information about prediction mode information (e.g., transition flag, merge flag, or mode index, etc.) and information about motion information as information related to the prediction process. The information about motion information may include candidate selection information (e.g., merge index, MVP flag, or MVP index) as information derived from the derivation of motion vectors. In addition, the information about motion information may include information about MVD and / or reference image index information.

[0158] Additionally, motion information may include indications of whether L0, L1, or bidirectional prediction is applied. Residual information is about the residual samples. Residual information may include information about the quantized transform coefficients applied to the residual samples.

[0159] The output bitstream can be stored in a (digital) storage medium and transmitted to a decoding device, or transmitted to a decoding device via a network.

[0160] Furthermore, as mentioned above, the encoding device can generate a reconstructed image (including reconstructed samples and reconstructed blocks) based on reference samples and residual samples. This is to derive the same prediction result as that performed by the decoding device, thus improving encoding efficiency. Therefore, the encoding device can store the reconstructed image (or reconstructed samples or reconstructed blocks) in memory and use the reconstructed image as a reference image. In-loop filtering can also be applied to the reconstructed image as described above.

[0161] Figure 10 and Figure 11 The diagram illustrates a video / image decoding method based on inter-frame prediction according to an embodiment of the present disclosure and an inter-frame prediction unit in a decoding apparatus according to an embodiment of the present disclosure.

[0162] Reference Figure 10 and Figure 11 The decoding device can perform operations corresponding to those performed by the encoding device. The decoding device can perform predictions on the current block based on the received prediction information and derive prediction samples.

[0163] S1001 to S1003 can be executed by the inter-frame prediction unit 260 of the decoding device, and the residual information of S1004 can be obtained from the bitstream by the entropy decoder 210 of the decoding device. The residual processing unit of the decoding device can derive the residual samples of the current block based on the residual information. Specifically, the dequantizer 220 of the residual processing unit derives the transform coefficients by performing dequantization based on the quantized transform coefficients derived based on the residual information, and the inverse transformer 230 of the residual processing unit can derive the residual samples of the current block by performing an inverse transform on the transform coefficients. S1005 can be executed by the adder 235 or the reconstruction unit of the decoding device.

[0164] Specifically, the decoding device can determine the prediction mode of the current block based on the received prediction information (S1001). The decoding device can determine which inter-frame prediction mode to apply to the current block based on the prediction mode information in the prediction information.

[0165] For example, a merge flag can be used to determine whether to apply a merge mode or (A)MVP mode to the current block. Alternatively, a mode index can be used to select one of a variety of inter-frame prediction mode candidates. Inter-frame prediction mode candidates may include skip mode, merge mode, and / or (A)MVP mode, or may include a variety of inter-frame prediction modes as described below.

[0166] The decoding device derives motion information for the current block based on the determined inter-frame prediction mode (S1002). For example, when applying a skip mode or a merge mode to the current block, the decoding device can configure a merge candidate list as described below and select one of the merge candidates included in the merge candidate list. The selection can be performed based on selection information (merge index). The motion information for the current block can be derived using the motion information of the selected merge candidate. The motion information of the selected merge candidate can be used as the motion information for the current block.

[0167] As another example, when applying the (A)MVP mode to the current block, the decoding device can configure an (A)MVP candidate list as described below, and use the motion vector of the selected MVP candidate from the motion vector prediction sub-MVP candidates included in the (A)MVP candidate list as the MVP of the current block. Selection can be performed based on selection information (MVP flag or MVP index). In this case, the MVD of the current block can be derived based on information about the MVD, and the motion vector of the current block can be derived based on the MVP and MVD of the current block. Additionally, the reference image index of the current block can be derived based on reference image index information. The image indicated by the reference image index in the reference image list of the current block can be derived as the reference image referenced for inter-frame prediction of the current block.

[0168] Furthermore, the motion information of the current block can be derived without the candidate list configuration described below, and in this case, the motion information of the current block can be derived according to the process disclosed in the prediction mode described below. In this case, the candidate list configuration can be omitted.

[0169] The decoding device can generate a prediction sample for the current block based on the motion information of the current block (S1003). In this case, a reference image can be derived based on the reference image index of the current block, and the prediction sample for the current block can be derived by using the sample of the reference block indicated by the motion vector of the current block on the reference image. In this case, as described below, in some cases, a filtering process for all or some of the prediction samples in the prediction sample of the current block can also be performed.

[0170] For example, the inter-frame prediction unit 260 of the decoding device may include a prediction mode determination unit 261, a motion information derivation unit 262, and a prediction sample derivation unit 263. The prediction mode determination unit 261 can determine the prediction mode of the current block based on the received prediction mode information, the motion information derivation unit 262 can derive the motion information (motion vector and / or reference image index) of the current block based on information about the received motion information, and the prediction sample derivation unit 263 can derive the prediction samples of the current block.

[0171] The decoding device generates residual samples for the current block based on the received residual information (S1004). The decoding device can generate reconstructed samples for the current block based on the predicted samples and residual samples, and generate a reconstructed image based on the generated reconstructed samples (S1005). Thereafter, an in-loop filtering process can be applied to the reconstructed image as described above.

[0172] As described above, the inter-frame prediction process may include an inter-frame prediction mode determination step, a motion information derivation step depending on the determined prediction mode, and a prediction execution (prediction sample generation) step based on the derivation of motion information.

[0173] Determine inter-frame prediction mode

[0174] Various inter-frame prediction modes can be used to predict the current block in an image. For example, various modes can be used, including merge mode, skip mode, MVP mode, affine mode, etc. Decoder-side motion vector correction (DMVR) mode, adaptive motion vector resolution (AMVR) mode, etc., can also be used as auxiliary modes. Affine mode can be called affine motion prediction mode. MVP mode can be called advanced motion vector prediction (AMVP) mode.

[0175] The encoding device can signal the decoding device with prediction mode information indicating the inter-frame prediction mode of the current block. This prediction mode information can be included in the bitstream and received by the decoding device. The prediction mode information may include index information indicating one of several candidate modes. Alternatively, the inter-frame prediction mode can be indicated by hierarchical signaling with flag information. In this case, the prediction mode information may include one or more flags.

[0176] For example, a skip mode can be indicated by signaling a transition flag, and a merge mode can be indicated by signaling a merge flag when skip mode is not applied, indicating whether MVP mode should be applied. Alternatively, a flag for additional differentiation can be signaled when MVP mode is not applied. An affine mode can be signaled as an independent mode or as a subordinate mode to a merge or MVP mode. For example, an affine mode can be configured as a candidate in a merge candidate list or an MVP candidate list as described below.

[0177] Motion information is derived from inter-frame prediction modes.

[0178] Inter-frame prediction can be performed using motion information from the current block. The encoding device can derive optimal motion information for the current block through a motion estimation process. For example, the encoding device can search for highly correlated similar reference blocks in a predetermined search range within a reference image, using original blocks from the original image, on a fractional-pixel basis, and derive motion information from the searched reference blocks. Block similarity can be derived based on differences between phase-based sample values. For example, block similarity can be calculated based on the SAD (Self-Average Difference) between the current block (or its template) and a reference block (or its template). In this case, motion information can be derived based on the reference block with the minimum SAD in the search region. The derived motion information can be signaled to the decoding device based on the inter-frame prediction mode using various methods.

[0179] Merge mode and skip mode

[0180] Figure 12 This is a diagram used to describe adjacent blocks used in merge mode or skip mode as an implementation of this disclosure.

[0181] When the merge mode is applied, the motion information of the current prediction block is not sent directly, but is inferred by using the motion information of neighboring prediction blocks. Therefore, a flag indicating the use of the merge mode and a merge index indicating which neighboring prediction block was used are sent to indicate the motion information of the current prediction block.

[0182] The encoder can search for candidate blocks to merge in order to derive motion information for the current predicted block and execute a merging pattern. For example, up to five candidate blocks can be used, but this disclosure is not limited to this. Additionally, a maximum number of candidate blocks can be transmitted in the slice header (or tile group header), and this disclosure is not limited to this. After finding candidate blocks, the encoder can generate a list of candidate blocks and select the candidate block with the lowest cost as the final candidate block.

[0183] This disclosure provides various implementations for merging candidate blocks that constitute a merge candidate list.

[0184] As a list of merge candidates, for example, five merge candidate blocks could be used. For instance, four spatial merge candidates and one temporal merge candidate could be used. As a concrete example, in the case of spatial merge candidates, Figure 12 The blocks illustrated in the example can be used as space merge candidates.

[0185] Figure 13 This is a flowchart illustrating a method for configuring and merging candidate lists according to an embodiment of the present disclosure.

[0186] Reference Figure 13 The encoding device (encoder / decoder) inserts spatial merging candidates derived by searching spatial neighboring blocks of the current block into the merging candidate list (S1301). For example, spatial neighboring blocks may include the lower-left neighboring block, left neighboring block, upper-right neighboring block, upper neighboring block, and upper-left neighboring block of the current block. However, this is just an example, and in addition to spatial neighboring blocks, supplementary neighboring blocks including right neighboring blocks, lower neighboring blocks, lower-right neighboring blocks, etc., can also be used as spatial neighboring blocks. The encoding device can derive available blocks by searching spatial neighboring blocks based on priority, and derive spatial merging candidates from the motion information of detected blocks. For example, the encoder and decoder can search in the order of A1, B1, B0, A0, and B2. Figure 12 The example shows five blocks, and the available candidates are indexed sequentially, and the indexed candidates are configured to merge the candidate list.

[0187] The encoding device inserts time-nearest blocks, derived by searching for time-nearest blocks of the current block, into the merge candidate list (S1302). Time-nearest blocks can be located on a reference picture, i.e., on a picture different from the current picture where the current block is located. The reference picture where the time-nearest blocks are located can be called the juxtaposed picture or the col picture. Time-nearest blocks can be searched in the order of the lower-right neighboring block and the lower-right center block of the juxtaposed block of the current block on the col picture.

[0188] In addition, when motion data compression is applied, for each predetermined storage unit, specific motion information can be stored as representative motion information in the col picture. In this case, it is not necessary to store the motion information of all blocks in the predetermined storage unit. As a result, a motion data compression effect can be obtained. In this case, the predetermined storage unit can be predetermined for each 16×16 sample unit or 8×8 sample unit, or the size information of the predetermined storage unit can be signaled from the encoder to the decoder. When motion data compression is applied, the motion information of a temporally neighboring block can be replaced with the representative motion information of the predetermined storage unit in which the temporally neighboring block is located.

[0189] In other words, in this case, in terms of implementation, a temporal merge candidate can be derived based on the motion information of a prediction block that covers a position that has undergone arithmetic right shift and then arithmetic left shift by a predetermined value based on the coordinates (upper left sample position) of temporally neighboring blocks other than the prediction block at the coordinates of the temporally neighboring block. For example, when the predetermined storage unit is a 2n×2n sample unit, if the coordinates of the temporally neighboring block are (xTnb, yTnb), the motion information of the prediction block at the position ((xTnb>>n)<<n), (yTnb>>n)<<n)), i.e., the corrected position, can be used for the temporal merge candidate.

[0190] Specifically, for example, when the predetermined storage unit is a 16×16 sample unit, if the coordinates of the temporally neighboring block are (xTnb, yTnb), the motion information of the prediction block at the position ((xTnb>>4)<<4), (yTnb>>4)<<4)), i.e., the corrected position, can be used for the temporal merge candidate. Alternatively, for example, when the predetermined storage unit is an 8×8 sample unit, if the coordinates of the temporally neighboring block are (xTnb, yTnb), the motion information of the prediction block at the position ((xTnb>>3)<<3), (yTnb>>3)<<3)), i.e., the corrected position, can be used for the temporal merge candidate.

[0191] The encoding device can check whether the current number of merge candidates is less than the maximum number of merge candidates (S1303). The maximum number of merge candidates can be predefined or signaled from the encoder to the decoder. For example, the encoder can generate information about the maximum number of merge candidates, encode the generated information, and transmit the encoded information in the form of a bitstream to the decoder. When the maximum number of merge candidates is full, subsequent candidate addition processing cannot be performed.

[0192] As a result of the check, when the current number of merge candidates is less than the maximum number of merge candidates, the encoding device will insert additional merge candidates into the merge candidate list (S1304). For example, additional merge candidates may include ATMVP, combined bidirectional prediction merge candidates (when the slice type of the current slice is type B), and / or zero vector merge candidates.

[0193] As a result of the check, if the current number of merge candidates is not less than the maximum number of merge candidates, the encoding device can terminate the configuration of the merge candidate list. In this case, the encoder can select the best merge candidate from the merge candidates constituting the merge candidate list based on the rate distortion (RD) cost, and can signal selection information (e.g., merge index) indicating the selected merge candidate to the decoder. The decoder can select the best merge candidate based on the merge candidate list and the selection information.

[0194] The motion information of the selected merging candidates can be used as the motion information of the current block, and the predicted sample of the current block can be derived based on the motion information of the current block as described above. The encoder can derive the residual sample of the current block based on the predicted sample and signal the residual information of the residual sample to the decoder. The decoder can generate reconstructed samples based on the residual samples derived based on the residual information and the predicted samples, and generate a reconstructed image based on the reconstructed samples generated as described above.

[0195] When the skip mode is applied, the motion information of the current block can be derived using the same method as when the merge mode is applied. However, when the skip mode is applied, the residual signal of the corresponding block is omitted, and as a result, the predicted sample can be directly used as the reconstructed sample.

[0196] MVP mode

[0197] Figure 14 This is a flowchart illustrating a method for configuring and merging candidate lists according to an embodiment of the present disclosure.

[0198] When applying the Motion Vector Prediction (MVP) pattern, this can be achieved by using reconstructed spatial neighbor blocks (e.g., the above). Figure 12 The motion vectors of the spatially neighboring blocks (as described in the diagram) and / or the motion vectors corresponding to the temporally neighboring blocks (or Col blocks) are used to generate a list of motion vector predictor (MVP) candidates. In other words, the motion vectors of the reconstructed spatially neighboring blocks and / or the motion vectors corresponding to the temporally neighboring blocks can be used as motion vector predictor candidates.

[0199] Information about the prediction can include selection information (e.g., MVP flag or MVP index) of the best motion vector prediction candidate selected from the motion vector prediction candidate list. In this case, the predictor can select the motion vector predictor for the current block from the motion vector prediction candidate list using the selected information. The predictor of the encoding device can obtain the motion vector difference (MVD) between the motion vector of the current block and the motion vector predictor, encode the obtained MVD, and output the encoded MVD as a bitstream. In other words, the MVD can be obtained by subtracting the motion vector predictor from the motion vector of the current block. In this case, the predictor of the decoding device can obtain the motion vector difference included in the prediction information and derive the motion vector of the current block by adding the motion vector difference to the motion vector predictor. The predictor of the decoding device can use the prediction information to obtain or derive the reference picture index of the indicator reference picture. For example, it can be as follows: Figure 14 The example demonstrates the configuration of the motion vector prediction sub-candidate list.

[0200] Advanced Time Motion Vector Prediction (ATMVP)

[0201] Figure 15 This is a diagram illustrating a method for deriving advanced time motion vector prediction (ATMVP) candidates as an implementation of this disclosure.

[0202] Reference Figure 15 ATMVP is a method that derives the motion information of sub-blocks of a coding unit based on the motion information of juxtaposed blocks of temporally neighboring images. Therefore, it can enhance the performance of Temporal Motion Vector Prediction (TMVP) and reduce the complexity in the general or worst-case scenarios. In this disclosure, ATMVP may be referred to as the Sub-Block-Based Temporal Merging Candidate SbTMVP.

[0203] According to embodiments of this disclosure, the following process can be used to derive ATMVP.

[0204] First, when a neighboring coding unit is available and its motion vector differs from the motion vector in the current candidate list, the encoder / decoder can add a motion vector from the spatially neighboring coding unit. As an example, this process can be performed as described above. Figure 12 The sequential execution of A1, B1, B0, A0, and B2 is illustrated in the example. As another example, to improve complexity, ATMVP can be derived in this process using only the motion vectors of the blocks at fixed positions (e.g., position A1).

[0205] The encoder / decoder can use the first motion vector candidate from No available spatial candidates to determine the location for deriving motion information for the juxtaposed image and each sub-block. Here, No represents the number of available spatial candidates. If No is 0, the motion information for each sub-block can be derived using the juxtaposed image and the juxtaposed position with motion of 0.

[0206] When using multiple reference images, the juxtaposed images of different coding units in ATMVP may differ from each other. The fact that different coding units in the current image have different juxtaposed images for deriving ATMVP means that motion information fields from multiple reference images should be derived, and this is not preferable because it increases storage bandwidth.

[0207] Therefore, this disclosure provides a simpler design for using the same juxtaposed image when deriving ATMVP. For example, a method for using the same juxtaposed image can be defined in the slice (or tile group) header, but this disclosure is not limited thereto. As an example, at the block level, when the reference image of a neighboring block A differs from the juxtaposed image, the motion vector of the neighboring block A can be scaled based on a time-motion vector scaling method. Additionally, the scaled motion vector of the neighboring block A can be used in the ATMVP.

[0208] Figure 16 This is a diagram illustrating a method for deriving advanced time motion vector prediction (ATMVP) candidates as an implementation of this disclosure.

[0209] Reference Figure 16 In embodiments of this disclosure, since the TMVP using the motion vector of the time neighboring block (or colPB) at the lower right block of the current block or at the center of the current block does not reflect intra-frame image motion, the encoder / decoder can use the motion vector of colPB at the location indicated by the motion vector of the neighboring block as the MVP.

[0210] For example, encoders / decoders can be like Figure 16 The illustrated merge candidate configuration checks blocks in the same order, while finding the motion vector of the first available space neighboring block. Additionally, the position indicated by the motion vector in the reference image can be derived as col-PB (i.e., ATMVP candidate).

[0211] Alternatively, motion vectors can be used as motion vectors for corresponding blocks on a per-sub-block basis. In this case, when there is no motion vector in a particular sub-block, the motion vector of the center block located at the center of the corresponding block can be used as the motion vector of the unavailable sub-block and stored as a representative motion vector.

[0212] Time motion vector data storage reduced

[0213] Embodiments of this disclosure propose a method for reducing temporal motion vector storage based on compressed spatial candidate motion vector data used for temporal motion vector data.

[0214] Figure 17 and Figure 18 This is a diagram illustrating a method for compressing time motion vector data and the location of spatial candidates therefor, according to an embodiment of the present disclosure.

[0215] Reference Figure 17 In embodiments of this disclosure, when spatial candidates are predicted via inter-frame prediction, the motion vectors of the spatial candidates can be configured as default motion vectors for compression. For example, up to five spatial candidates can be used as reference temporal motion information for deriving the default temporal motion vector. As an implementation, it can be as follows... Figure 18 The example shows the configuration of five spatial candidates.

[0216] Additionally, temporal motion vector data can be compressed based on spatial candidate motion vectors. Figure 17 The image illustrates the order of candidate search spaces. Candidate spaces can be identified based on the order of center block C, top-left block TL, top-right block TR, bottom-left block BL, and bottom-right block BR. This is merely one implementation method, and the disclosure is not limited thereto; other combinable orders may be applied.

[0217] First, the encoder / decoder can check whether center block C is inter-predicted. If center block C is inter-predicted, the encoder / decoder can configure the motion vector of center block C as the default for predicting motion vectors.

[0218] If the center block C is not inter-predicted, the encoder / decoder can check if the top-left block TL is inter-predicted. If the top-left block TL is inter-predicted, the encoder / decoder can configure the motion vector of the top-left block TL as the default for predicting motion vectors.

[0219] If the top-left block TL is not inter-predicted, the encoder / decoder can check if the top-right block TR is inter-predicted. If the top-right block TR is inter-predicted, the encoder / decoder can configure the motion vector of the top-right block TR to the default used for predicting motion vectors.

[0220] If the top-right block TR is not inter-predicted, the encoder / decoder can check if the bottom-left block BL is inter-predicted. If the bottom-left block BL is inter-predicted, the encoder / decoder can configure the motion vectors of the bottom-left block BL to be the default for predicting motion vectors.

[0221] If the lower left block (BL) is not inter-predicted, the encoder / decoder can check if the lower right block (BR) is inter-predicted. If the lower right block (BR) is inter-predicted, the encoder / decoder can configure the motion vectors of the lower right block (BR) to be the default for predicting motion vectors.

[0222] If the bottom right block BR ​​is not predicted inter-frame, the encoder / decoder can configure the intra-frame mode as the default.

[0223] Through this process, the encoder / decoder can compress the default motion vectors into motion information.

[0224] Implementation of ATMVP based on adaptive sub-block size

[0225] Implementations of this disclosure present a method for executing ATMVP based on adaptive sub-block sizes. For example, sub-block sizes for deriving ATMVP can be adaptively applied at the slice level.

[0226] Furthermore, if ATMVP motion information is derived in units of 4×4 blocks, the following problem may exist: motion derivation and motion compensation are performed in every 4×4 sub-blocks within an ATMVP coding unit.

[0227] To address this issue, the encoder can signal the decoder to indicate a default sub-block size for sequence-level ATMVP motion derivation.

[0228] As another example, when using the default sub-block size in the current slice, a flag can be signaled at the image or slice level. If the flag is false, the ATMVP sub-block size can be signaled separately in the slice header.

[0229] Implementation methods for deriving the restricted region of juxtaposed blocks

[0230] In this disclosure, the region for the juxtaposition block used in ATMVP may include an N×N block of one column from the current CTU and the juxtaposition image. For example, the N×N block may be a 4×4 block, but this disclosure is not limited thereto.

[0231] The ATMVP juxtaposition block identified by the motion vector of the merging candidate is located outside the restricted region. The ATMVP juxtaposition block can be moved to be located within the restricted region. For example, the ATMVP juxtaposition block can be moved to be located on the nearest boundary within the restricted region.

[0232] Derivation of an implementation method for sub-block-based time merging candidates

[0233] In embodiments of this disclosure, the encoder / decoder can add motion information of juxtaposed blocks (or juxtaposed sub-blocks) in a juxtaposed image, specified based on motion information of spatially neighboring blocks, as sub-block-based temporal merging candidates to the sub-block merging candidate list.

[0234] In this disclosure, the motion information of spatially neighboring blocks can be referred to as a temporal motion vector. As an implementation, when the width and height of the current coded block are equal to or greater than a predetermined specific size, the encoder / decoder can derive temporal merging candidates based on sub-blocks. For example, the predetermined specific size can be 8.

[0235] As an implementation, the encoder / decoder can configure the motion information of a first spatial candidate among the available spatial candidates as a temporal motion vector. For example, the encoder / decoder can search for available spatial candidates in the order of A1, B1, B0, and A0. In this case, the encoder / decoder can configure a spatial candidate among the available spatial candidates whose reference image is the same as the juxtaposed image as a temporal motion vector. As another example, the encoder / decoder can check whether a fixed-position spatial candidate is available, and when a fixed-position spatial candidate is available, the encoder / decoder can configure the motion vector of the corresponding spatial candidate as a temporal motion vector. For example, a fixed-position spatial candidate can be configured as a block at position A1.

[0236] Additionally, the encoder / decoder can specify the position of juxtaposed blocks in the juxtaposed images by using temporal motion vectors. As an example, Equation 1 above can be used.

[0237] [Formula 1]

[0238] xColCb=Clip3(xCtb,Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xColCtrCb+(tempMv[0]> >4))

[0239] yColCb=Clip3(yCtb,Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),yColCtrCb+(tempMv[1]> >4))

[0240] Here, (xColCtrCb, yColCtrCb) represents the position of the top left sample in the juxtaposed coding block, including the top right sample at the center position, and tempMv represents the time motion vector.

[0241] Additionally, the encoder / decoder can determine the positions used to derive motion information for each sub-block in the current encoded block, on a sub-block basis. In an implementation, the positions of juxtaposed sub-blocks in a juxtaposed image can be derived using Equation 2 below.

[0242] [Equation 2]

[0243] xColSb=Clip3(xCtb,Min(CurPicWidthInSamplesY-1,xCtb+(1<<CtbLog2SizeY)+3),xSb+(tempM v[0]> >4))

[0244] yColSb=Clip3(yCtb,Min(CurPicHeightInSamplesY-1,yCtb+(1<<CtbLog2SizeY)-1),ySb+(temp Mv[1]> >4))

[0245] Here, (xSb, ySb) represents the position of the current sub-block.

[0246] In this implementation, when the current juxtaposed subblock is unavailable, the encoder / decoder can use the motion information of the juxtaposed block specified by the time motion vector.

[0247] Figure 19 This diagram illustrates a method for configuring a merging candidate list according to conventional image compression techniques, which can be applied as an embodiment of this disclosure.

[0248] In conventional image compression techniques (e.g., HEVC), when a merge mode is applied, the encoder / decoder adds candidates sequentially to the candidate list using spatial candidates, temporal candidates, combined bidirectional prediction candidates, and zero-motion candidates until the total number of candidates is 5, in order to configure the candidate list.

[0249] In this implementation, the encoder / decoder can add a predetermined number of candidates for each candidate. The encoder / decoder considers each candidate sequentially, but if... Figure 19 As illustrated, the candidate generation process can be terminated when the maximum number is met. For example, the encoder / decoder can consider up to four candidates as spatial candidates. Furthermore, the encoder / decoder can consider up to five candidates as combined bidirectional prediction candidates. Additionally, the encoder / decoder can consider up to five candidates as zero-motion candidates. In this case, as spatial candidates, it can be determined according to the above... Figure 12 The order of A1, B1, B0, A0 and B2 shown in the example considers a maximum of four candidates.

[0250] Implementation Method 1

[0251] Embodiments of this disclosure provide a method for generating corrected merge candidates by using merge offsets. As an embodiment, the encoder / decoder can generate corrected merge candidates by adding motion vector offsets to existing merge candidates in the merge candidate list.

[0252] In this disclosure, a candidate (or merged candidate) in the candidate list (merged candidate list) may be referred to as a seed candidate. However, this disclosure is not limited thereto. For example, a seed candidate may be referred to as a candidate added to the candidate list (merged candidate list), an added candidate, a candidate configured by the candidate list, a configured candidate, etc.

[0253] Furthermore, in embodiments of this disclosure, a predefined number of seed candidates can be used to generate corrected merge candidates. In this disclosure, for ease of description, the number of seed candidates used to generate corrected candidates can be represented as n_seed. For example, when n_seed = 1, the encoder / decoder can select either the first candidate in the merge candidate list or a candidate at a predefined position as a seed candidate for generating corrected candidates. Alternatively, for example, when n_seed = 1 and the predefined position is... Figure 12 When the value is A0, the encoder / decoder can use the candidate block at position A0 as a seed candidate.

[0254] Alternatively, for example, when n_seed = n, the encoder / decoder can use n candidates as seed candidates in the merge candidate list in order (e.g., merge index order). Alternatively, for example, when n_seed = n, candidates at predefined n positions can be used as seed candidates. The encoder / decoder then determines the seed candidates and adds motion vector offsets to each seed candidate to generate a corrected candidate. Furthermore, in this disclosure, the motion vector offsets added to generate the corrected merge candidate are not limited to their names. For example, the motion vector offset may be referred to as merged motion vector offset, motion vector difference, motion vector difference offset, merged motion vector difference, etc.

[0255] Furthermore, according to embodiments of this disclosure, the generated candidate (or merged candidate) may be referred to as a modified candidate (or modified merged candidate). However, this disclosure is not limited thereto. For example, a modified candidate may be referred to as a merged candidate, a general merged candidate, a candidate (or merged candidate) with added motion vector offset, a candidate (or merged candidate) with added motion vector difference, etc.

[0256] Figure 20 This is a flowchart illustrating a method for generating modified merge candidates as an embodiment of the present disclosure.

[0257] Reference Figure 20 In describing the embodiments of this disclosure, the decoder is described primarily for ease of description, but the candidate list generation method of this disclosure can also be applied equivalently to the encoder.

[0258] The decoder configures (or generates) a merging candidate list by using motion information from spatial and temporal candidate blocks adjacent to the current block (S2001). As an implementation, the encoder / decoder can apply the above... Figures 12 to 18 The method described in [the document] configures the merge candidate list.

[0259] The decoder selects a predefined specific number of merge candidates from the merge candidate list configured in step S2001 above for the candidate derivation of the correction (S2002). In this disclosure, for ease of description, the case where the specific number is 2 is primarily described, but this disclosure is not limited to this, and one or three or more predefined numbers of candidates can be selected. As an example, the decoder can select a first merge candidate and a second merge candidate from the merge candidate list. In other words, the decoder can select two merge candidates in the order in which the merge indices are assigned in the merge candidate list. This specific number can be predefined in both the encoder and decoder and is signaled from the encoder to the decoder. If a specific number is signaled, it can be signaled through a sequence parameter set, image parameter set, slice header (or tile group header), etc.

[0260] The decoder determines (or selects) a predefined number of merge candidates for inter-frame prediction applied to the current block (S2003). As an implementation, the decoder may parse a first syntax element indicating a predefined number of candidates for inter-frame prediction applied to the current block. That is, the decoder may determine, based on the syntax element, a merge candidate for deriving a modified first or second merge candidate.

[0261] The decoder generates (or derives) a modified merging candidate by adding a motion vector offset to the merging candidate motion vector determined in step S2003 above (S2004). In this case, the encoder can signal the motion vector offset, and the encoder and decoder can derive the same value.

[0262] In implementations, the motion vector offset can have integer pixel precision and fractional pixel precision. For example, the precision (or basic unit) of the motion vector offset can have values ​​such as 1, 1 / 2, 1 / 4, 1 / 8, or 1 / 16. Alternatively, for example, the precision of the motion vector offset can have the same precision as the motion vectors merging candidate motion vectors.

[0263] Additionally, in implementations, the precision of the motion vector offset can be defined (or signaled) using a more advanced syntax. For example, the precision of the motion vector offset can be signaled from the encoder to the decoder via sequence parameter sets, image parameter sets, slice headers (or tile group headers), etc.

[0264] The motion vector offset can have values ​​that are integer multiples of the basic unit, and can be represented by Equation 3 below.

[0265] [Formula 3]

[0266] MV offset ={(v x v y )v x =p*i,v y =p*j}

[0267] In Equation 3, i and j are integers (natural numbers) other than 0. In addition, p represents the predefined basic unit of motion vector offset.

[0268] The decoder generates a prediction block by using the motion information of the modified merged candidates (S2005).

[0269] In implementation, modified candidates (or modified merged candidates) can be considered in order of proximity to the seed candidates in terms of distance (or in order of smaller absolute values ​​of motion vector offsets). As mentioned above, the number of modified candidates used can be defined in a more advanced syntax, and in this case, the number of modified candidates used can be used within a range not exceeding the maximum number of available merged candidates in the merged candidate list.

[0270] Additionally, in this implementation, the decoder can add motion information with added motion vector offsets to the merge candidate list. When a corrected candidate is considered for merging, a pruning check (or duplicate check) is performed to add the corrected candidate to the merge candidate list only if it is different from any candidate already existing in the merge candidate list. Alternatively, the decoder can add the corrected candidate to the merge candidate list without performing a pruning check, taking into account decoding complexity.

[0271] Additionally, in the implementation, the decoder can consider the correction candidates in a predefined specific order. In this case, the positions used to consider the correction candidates can be one, two, or more positions. For example, the specific order (or position) for considering the correction candidates mentioned above could be 1) the blocks at positions A1, B1, B0, and A0 in the spatial candidates (or spatially adjacent blocks) in sequence (see...). Figure 12 1) after confirming spatial candidates, 2) after confirming spatial candidates and temporal candidates, 3) after confirming spatial candidates and temporal candidates, 4) after confirming spatial candidates, temporal candidates and combined candidates, or 5) after confirming spatial candidates, temporal candidates and zero-motion candidates.

[0272] Alternatively, for example, when the number of candidate positions to be considered for correction is two or more, the decoder may consider n corrected candidates after confirming spatial candidates (blocks A1, B1, B0, and A0), and additionally consider p corrected candidates after confirming temporal candidates. In this case, the decoder may consider candidates that are the same as the seed candidate at each position, and also consider seed candidates that are different at each position.

[0273] Additionally, in this implementation, if a merged candidate list cannot be configured before generating the revised candidates, the decoder may not configure the revised candidates. Alternatively, the decoder may determine the zero motion vector as a seed candidate and generate the revised candidates.

[0274] Figure 21 This is an illustration of a motion vector offset for generating corrected merging candidates as an embodiment of the present disclosure.

[0275] exist Figure 21 In this context, it is assumed that the motion vector offset is determined in units of 1 / 2 pixels. However, this disclosure is not limited to this, and as mentioned above, the precision (or basic unit) of the motion vector offset can have values ​​such as 1, 1 / 2, 1 / 4, 1 / 8, or 1 / 16. Furthermore, in... Figure 21 In this context, it is assumed that the maximum number of available offsets (or modified candidates) is 8. However, this disclosure is not limited to this, and multiple pixels (integer pixels or subpixels) adjacent to the pixel specified by the merge candidate in the merge candidate list can be considered as modified candidates.

[0276] Reference Figure 21 The dashed lines represent a 1 / 2 pixel grid, and the point at the center is the first seed candidate (or indicates the position of the motion vector of the first seed candidate). As an implementation, the four modified candidates closest to the seed candidate marked by X can be preferentially considered. The four modified candidates marked by the second closed square can also be considered. In the example, modified candidates out of a total of eight candidates, not exceeding the maximum number of merged candidates, can be considered as merged candidates. Alternatively, pixels (integer pixels or sub-pixels) adjacent to the seed candidate (or indicating the position of the motion vector of the seed candidate) or motion vector offsets indicating the corresponding positions can be sent from the encoder to the decoder.

[0277] Implementation Method 2

[0278] Embodiments of this disclosure provide a method for generating modified merge candidates using a weighted average. As an embodiment, the encoder / decoder can generate (or derive) modified merge candidates by performing a weighted average on a specific number of merge candidates in a merge candidate list.

[0279] In embodiments of this disclosure, a predefined number of seed candidates can be used to generate modified merge candidates. For ease of description, this disclosure primarily focuses on the case where the specific number is 2, but it is not limited to this, and one, three, or more predetermined numbers of candidates can be selected. For example, the encoder / decoder can sequentially use n candidates as seed candidates within the merge candidate list.

[0280] In this disclosure, a candidate (or merged candidate) in the candidate list (merged candidate list) may be referred to as a seed candidate. However, this disclosure is not limited thereto. For example, a seed candidate may be referred to as a candidate added to the candidate list (merged candidate list), an added candidate, a candidate configured by the candidate list, a configured candidate, etc.

[0281] Furthermore, according to embodiments of this disclosure, the generated candidate (or merged candidate) may be referred to as a modified candidate (or modified merged candidate). However, this disclosure is not limited thereto. For example, a modified candidate may be referred to as a merged candidate, a general merged candidate, a candidate (or merged candidate) with added motion vector offset, a candidate (or merged candidate) with added motion vector difference, etc.

[0282] As an example, the modified merge candidate can be derived using Equation 4 below.

[0283] [Formula 4]

[0284] Cand refined = (1-w)*seed_1st+w*seed_2nd

[0285] In Equation 4, seed_1st and seed_2nd represent the first and second seed candidates, respectively. Additionally, the value of w can be one of the values ​​represented by Equation 5 below.

[0286] [Formula 5]

[0287]

[0288]

[0289]

[0290]

[0291] That is, the decoder can generate a modified merge candidate by taking a weighted average of two predefined candidate pairs in the merge candidate list. For example, the predefined pair could be {(0,1),(0,2),(1,2),(0,3)(1,3),(2,3)}. Here, the numbers represent the merge index (or merge candidate order) in the merge candidate list. In an implementation, the weighted average motion vector can be calculated separately for the corresponding list of reference images (or reference list). When two motion vectors are available in a list of reference images, a weighted average can be performed on each motion vector even when the motion vectors indicate different reference images.

[0292] The method proposed in the implementation method can be applied only to the above. Figures 12 to 19 In the embodiments described herein and / or in Embodiment 1 above, taking the application of the method proposed in the embodiments to Embodiment 1 above as an example, the encoder / decoder can add the weighted average candidate proposed in the embodiments to the merging candidate list after configuring the merging candidate list using motion information from spatial candidate blocks and temporal candidate blocks. Additionally, the encoder / decoder can derive the corrected merging motion vector by adding motion vector offsets to the merging candidates in the merging candidate list.

[0293] Additionally, in one implementation, the encoder / decoder may only consider merging candidates that include the same reference image as seed candidates. Alternatively, the encoder / decoder may add a modified merging candidate to the merging candidate list using a motion vector scaled and weighted by a specific reference frame (e.g., a juxtaposed image). For example, a specific reference frame could be used for each reference frame, or the reference frame that appears most frequently in the surrounding blocks could be used.

[0294] Implementation Method 3

[0295] Embodiments of this disclosure provide a method for generating modified merge candidates by using averaging. As an embodiment, the encoder / decoder can generate (or derive) modified merge candidates by performing an average on a specific number of merge candidates in a merge candidate list.

[0296] In embodiments of this disclosure, a predefined number of seed candidates can be used to generate modified merge candidates. For ease of description, this disclosure primarily focuses on the case where the specific number is 2, but it is not limited to this and may select one, three, or more predetermined numbers of candidates. For example, the encoder / decoder may sequentially use n candidates as seed candidates within the merge candidate list. Alternatively, the selected seed candidates may be combinations that can be determined from an already determined merge candidate list. For example, when the number of candidates in the already determined merge candidate list is 3, the number of combinations of two seed candidates can be 3, and three modified candidates (or modified merge candidates) can be derived based on this.

[0297] Alternatively, in an implementation, the number of seed candidates can be n, and the value of n can be determined by the number of candidates that can be considered as seed candidates (i.e., the number of merge candidates in the already configured merge candidate list). For example, when the number of candidates that can be used as seed candidates is 3, both the cases of n=2 and n=3 can be considered.

[0298] In this disclosure, a candidate (or merged candidate) in the candidate list (merged candidate list) may be referred to as a seed candidate. However, this disclosure is not limited thereto. For example, a seed candidate may be referred to as a candidate added to the candidate list (merged candidate list), an added candidate, a candidate configured by the candidate list, a configured candidate, etc.

[0299] Furthermore, according to embodiments of this disclosure, the generated candidate (or merged candidate) may be referred to as a modified candidate (or modified merged candidate). However, this disclosure is not limited thereto. For example, a modified candidate may be referred to as a merged candidate, a general merged candidate, a candidate (or merged candidate) with added motion vector offset, a candidate (or merged candidate) with added motion vector difference, etc.

[0300] As an example, the modified merge candidate can be derived using Equation 6 below.

[0301] [Formula 6]

[0302]

[0303] In Equation 6, seed_1st and seed_2nd can represent the first seed candidate and the second seed candidate, respectively, and seed_1st and seed_2nd can be selected as a specific combination of candidates in the already determined list of merged candidates.

[0304] That is, the decoder can generate a modified merge candidate by averaging two predefined candidate pairs in the merge candidate list. For example, the predefined pair could be {(0,1),(0,2),(1,2),(0,3)(1,3),(2,3)}. Here, the numbers represent the merge index (or merge candidate order) in the merge candidate list. In an implementation, the averaged motion vector can be calculated separately for the corresponding list of reference images (or reference list). When two motion vectors are available in a list of reference images, the individual motion vectors can be averaged even when they indicate different reference images.

[0305] The method proposed in the implementation method can be applied only to the above. Figures 12 to 19 In the embodiments described herein and / or in Embodiment 1 above, taking the application of the method proposed in the embodiments to Embodiment 1 above as an example, the encoder / decoder can add the averaged candidate proposed in the embodiments to the merged candidate list after configuring the merged candidate list using motion information of spatial candidate blocks and temporal candidate blocks. Additionally, the encoder / decoder can derive the corrected merged motion vector by adding motion vector offsets to the merged candidates in the merged candidate list.

[0306] Additionally, in one implementation, the encoder / decoder may only consider merging candidates that include the same reference image as seed candidates. Alternatively, the encoder / decoder may add merging candidates to the merging candidate list as modified merging candidates, using motion vectors scaled to a specific reference frame (e.g., juxtaposed images) and averaged. For example, a specific reference frame could be used for each reference frame, or the reference frame that appears most frequently in the surrounding blocks could be used.

[0307] As described above, embodiments of the present disclosure have been described separately for ease of description, but the present disclosure is not limited thereto. That is, the embodiments described in embodiments 1 to 3 above can be performed independently, and one or more of the various embodiments can be combined and performed.

[0308] Figure 22 This is a flowchart illustrating a method for generating inter-frame prediction blocks according to an embodiment of the present disclosure.

[0309] Reference Figure 22 The decoder has been described primarily for ease of description, but this disclosure is not limited thereto, and the method for generating inter-frame prediction blocks according to embodiments of this disclosure can be similarly performed in the encoder and decoder.

[0310] The decoder configures the merge candidate list by using motion information of spatial and temporal candidates adjacent to the current block (S2201).

[0311] As described in Implementation 2 above, the decoder can add candidates generated by weighting predefined candidate pairs in a previously determined merge candidate list to the merge candidate list.

[0312] Furthermore, as described in Embodiment 3 above, the decoder can add candidates generated by averaging predefined candidate pairs in a previously determined merge candidate list to the merge candidate list.

[0313] The decoder resolves the first syntax element of the specific candidate applied to the inter-frame prediction of the current block from a predefined number of candidates in the merge candidate list (S2202).

[0314] As described in Embodiment 1 above, the decoder can parse the first syntax element of a specific candidate that is applied to the inter-frame prediction of the current block, which is one of the first and second candidates that indicate the order in which the merge indexes are assigned in the merge candidate list.

[0315] The decoder derives the corrected candidate by adding a motion vector offset to the candidate motion vector indicated by the first syntax element (S2203).

[0316] As described in Embodiment 1 above, the motion vector offset can be derived based on whether the motion vector offset has integer pixel precision or fractional pixel precision.

[0317] Furthermore, as described in Embodiment 1 above, the decoder can parse a second syntax element indicating the accuracy of the motion vector offset, and in this case, the second syntax element can be sent via a sequence parameter set, a picture parameter set, or a slice header.

[0318] Furthermore, as described in Embodiment 1 above, the motion vector offset can be derived based on the value obtained by multiplying the motion vector offset precision by a non-zero integer.

[0319] The decoder generates a prediction block for the current block based on the motion information of the current block (S2204).

[0320] Figure 23 This is a diagram illustrating an inter-frame prediction apparatus according to an embodiment of the present disclosure.

[0321] exist Figure 23 In this document, for ease of description, the inter-frame prediction unit is exemplified as a block, but it can be implemented as a component included in the encoder and / or decoder.

[0322] Reference Figure 23 The inter-frame prediction unit achieves the above. Figures 8 to 22The functions, processes, and / or methods proposed in the document. Specifically, the inter-frame prediction unit can be configured to include a merging candidate list configuration unit 2301, a first syntax element parsing unit 2302, a corrected candidate derivation unit 2303, and a prediction block generation unit 2304.

[0323] The merge candidate list configuration unit configures the merge candidate list by using motion information of spatial and temporal candidates adjacent to the current block.

[0324] As described in Embodiment 2 above, the merged candidate list configuration unit 2301 can add candidates generated by weighting the predefined candidate pairs in the previously determined merged candidate list to the merged candidate list.

[0325] Furthermore, as described in Embodiment 3 above, the merged candidate list configuration unit 2301 can add candidates generated by averaging predefined candidate pairs in a previously determined merged candidate list to the merged candidate list.

[0326] The first syntax element parsing unit 2302 parses the first syntax element of a specific candidate among a predefined number of candidates in the merge candidate list that is applied to the inter-frame prediction of the current block.

[0327] As described in Embodiment 1 above, the first syntax element parsing unit 2302 can parse the first syntax element of a specific candidate applied to the inter-frame prediction of the current block, indicating the order in which the first and second candidates are assigned merge indices in the merge candidate list.

[0328] The modified candidate derivation unit 2303 derives the modified candidate by adding a motion vector offset to the motion vector of the candidate indicated by the first syntax element.

[0329] As described in Embodiment 1 above, the motion vector offset can be derived based on whether the motion vector offset has integer pixel precision or fractional pixel precision.

[0330] Furthermore, as described in Embodiment 1 above, the modified candidate derivation unit 2303 can parse the second syntax element indicating the accuracy of the motion vector offset, and in this case, the second syntax element can be sent through a sequence parameter set, an image parameter set, or a slice header.

[0331] Furthermore, as described in Embodiment 1 above, the motion vector offset can be derived based on the value obtained by multiplying the motion vector offset precision by a non-zero integer.

[0332] The prediction block generation unit 2304 generates a prediction block for the current block by using the motion information of the current block.

[0333] Figure 24 An example of a video coding system using this disclosure is provided.

[0334] A video encoding system may include a source device and a receiving device. The source device may forward encoded video / image information or data to the receiving device in file or streaming format via digital storage media or a network.

[0335] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be configured as a stand-alone device or an external component.

[0336] Video sources can be acquired through processes such as capture, composition, or generation to obtain video / images. Video sources may include video / image capture devices and / or video / image generation devices. For example, a video / image capture device may include one or more cameras, a video / image archive comprising previously captured video / images, etc. A video / image generation device may include, for example, a computer, tablet, and smartphone, and may generate video / images, for example, (electronically). For instance, virtual video / images can be generated by a computer, and in this case, the video / image capture process can be replaced by a process that generates related data.

[0337] Encoding devices can encode input video / images. Encoding devices can perform a series of processes, including prediction, transformation, and quantization, for compression and coding efficiency.

[0338] A transmitter can forward encoded video / image information or data, output in bitstream format, to a receiver in a receiving device via digital storage media or a network, in file or streaming format. Digital storage media can include various media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter can include elements for generating media files in a predetermined file format and may include elements for transmission over a broadcast / communication network. The receiver can extract the bitstream and forward it to a decoding device.

[0339] Decoding devices can perform a series of processes, including inverse quantization, inverse transform, prediction, etc., corresponding to the operations of encoding devices, and decode video / images.

[0340] The renderer can render the decoded video / image. The rendered video / image can be displayed through the display unit.

[0341] Figure 25 This is a configuration diagram of a content streaming system as an embodiment of the present disclosure.

[0342] Reference Figure 25 The content streaming system using this disclosure may include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[0343] An encoding server is used to compress content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and then sends that bitstream to a streaming server. As another example, when multimedia input devices such as smartphones, cameras, and camcorders generate bitstreams directly, the encoding server can be omitted.

[0344] Bitstreams can be generated by applying the encoding method or bitstream generation method disclosed herein, and the streaming server can temporarily store bitstreams during the processing of sending or receiving bitstreams.

[0345] The streaming server sends multimedia data to the user's device via a web server based on a user request, and the web server acts as a medium for informing the user of services. When a user sends a request for a desired service to the web server, the web server forwards the request to the streaming server, and the streaming server sends the multimedia data to the user. Here, the content streaming system may include an additional control server, and in this case, the control server is used to control the commands / responses between devices in the content streaming system.

[0346] A streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from an encoding server, the streaming server can receive the content in real time. In this case, the streaming server can store the bitstream for a predetermined period of time to provide a smooth streaming service.

[0347] Examples of user equipment may include cellular phones, smartphones, laptops, digital broadcast terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation devices, touchscreen PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, and HMDs (head-mounted displays)), digital TVs, desktop computers, digital signage, etc.

[0348] In a content streaming system, each server can operate as a distributed server, and in this case, the data received by each server can be processed in a distributed manner.

[0349] The embodiments described in this disclosure can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units illustrated in the drawings can be implemented and executed on a computer, processor, microprocessor, controller, or chip.

[0350] Furthermore, the decoders and encoders disclosed herein can be included in multimedia broadcasting transmitting and receiving devices, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video chat devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, video-on-demand (VoD) service providers, over-the-top (OTT) video devices, internet streaming service providers, three-dimensional (3D) video devices, video telephony devices, and medical video devices, and can be used to process video signals or data signals. For example, OTT video devices can include game consoles, Blu-ray players, internet access TVs, home theater systems, smartphones, tablet PCs, and digital video recorders (DVRs).

[0351] Furthermore, the processing methods of this disclosure can be generated in the form of a program executed by a computer and can be stored in a computer-readable recording medium. Multimedia data with data structures according to this disclosure can also be stored in a computer-readable recording medium. A computer-readable recording medium includes all kinds of storage devices in which computer-readable data is stored. Computer-readable recording media can include, for example, Blu-ray discs (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disks, and optical data storage devices. Furthermore, computer-readable recording media include media implemented in the form of a carrier wave (e.g., transmitted via the Internet). Additionally, bitstreams generated using encoding methods can be stored in a computer-readable recording medium or transmitted via wired or wireless communication networks.

[0352] Furthermore, the embodiments of this disclosure can be implemented as a computer program product using program code. The program code can be executed by a computer according to the embodiments of this disclosure. The program code can be stored on a computer-readable medium.

[0353] In the embodiments mentioned above, the elements and features of this disclosure have been combined in a specific manner. Each of these elements or features may be considered optional unless otherwise explicitly described. Each of these elements or features may be implemented without combination with other elements or features. Furthermore, some of these elements and / or features may be combined to form embodiments of this disclosure. The order of operations described in the embodiments of this disclosure may be changed. Some elements or features of an embodiment may be included in another embodiment, or may be replaced by corresponding elements or features of another embodiment. It is apparent that embodiments may be constructed by combining claims that are not expressly referenced in the claims or may be included as new claims by amendments after filing the application.

[0354] Embodiments according to this disclosure can be implemented using various means (e.g., hardware, firmware, software, or combinations thereof). In the case of hardware implementation, embodiments of this disclosure can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0355] When implemented in firmware or software, embodiments of this disclosure can be implemented as modules, processes, or functions that perform the functions or operations mentioned above. Software code can be stored in memory and driven by a processor. The memory can be located internally or externally to the processor and can exchange data with the processor using various known means.

[0356] It will be apparent to those skilled in the art that this disclosure may be implemented in other specific forms without departing from its essential characteristics. Therefore, the specific embodiments should not be construed as limiting, but rather as illustrative in all respects. The scope of this disclosure should be determined through a reasonable analysis of the appended claims, and all modifications within the equivalent scope of this disclosure are included within its scope.

[0357] Industrial applicability

[0358] The preferred embodiments mentioned above have been disclosed for illustrative purposes, and those skilled in the art can improve, modify, substitute or add various other embodiments without departing from the technical spirit and scope of the present disclosure as disclosed in the appended claims.

Claims

1. A decoding device for image decoding, the decoding device comprising: Memory; as well as At least one processor connected to the memory, the at least one processor being configured to: Determine whether the merging mode is applied to the current block among multiple inter-frame prediction modes, wherein the multiple inter-frame prediction modes include the merging mode and the motion vector prediction MVP mode; Based on the determination that the merging mode is applied to the current block, a merging candidate list is configured under the merging mode based on motion information of spatial and temporal candidates for the current block; Obtain merged motion vector offset accuracy information related to whether the merged motion vector offset has fractional pixel accuracy, wherein the merged motion vector offset accuracy information is obtained from the image parameter set; Merging candidates are determined for deriving the corrected merging candidate based on the merging motion vector offset, wherein the merging candidate at position 0 of the merging candidate list is determined as the merging candidate for deriving the corrected merging candidate based on the merging motion vector offset; The motion information of the corrected merging candidate is derived based on the sum of the determined motion vector of the merging candidate and the merging motion vector offset derived based on the merging motion vector offset accuracy information; and The predicted block for the current block is generated based on the motion information of the corrected merging candidates. Specifically, after configuring the merged candidate list including the spatial candidates and the temporal candidates, merged candidates generated based on the average of predefined candidate pairs are added to the merged candidate list, and The predefined candidate pair is the pair of the candidate at position 0 and the candidate at position 1 in the merged candidate list.

2. The decoding device according to claim 1, wherein, The merged motion vector offset is derived based on the value obtained by multiplying the accuracy of the merged motion vector offset by a non-zero integer.

3. The decoding device according to claim 1, wherein, Merge candidates generated based on a weighted average of predefined candidate pairs are added to the merge candidate list.

4. An encoding device for image encoding, the encoding device comprising: Memory; as well as At least one processor connected to the memory, the at least one processor being configured to: Determine whether the merging mode is applied to the current block among multiple inter-frame prediction modes, wherein the multiple inter-frame prediction modes include the merging mode and the motion vector prediction MVP mode; Based on the determination that the merging mode is applied to the current block, a merging candidate list is configured under the merging mode based on motion information of spatial and temporal candidates for the current block; Merging candidates are determined for deriving the corrected merging candidate based on the merging motion vector offset, wherein the merging candidate at position 0 of the merging candidate list is determined as the merging candidate for deriving the corrected merging candidate based on the merging motion vector offset. The motion information of the corrected merging candidate is derived by summing the motion vector of the determined merging candidate and the offset of the merging motion vector derived based on the offset accuracy information of the merging motion vector. The predicted block for the current block is generated based on the motion information of the corrected merging candidates; and Video information is generated that includes merged motion vector offset accuracy information related to whether the merged motion vector offset has fractional pixel accuracy, wherein the merged motion vector offset accuracy information is included in the image parameter set of the video information. Specifically, after configuring the merged candidate list including the spatial candidates and the temporal candidates, merged candidates generated based on the average of predefined candidate pairs are added to the merged candidate list, and The predefined candidate pair is the pair of the candidate at position 0 and the candidate at position 1 in the merged candidate list.

5. The encoding device according to claim 4, wherein, The merged motion vector offset is derived based on the value obtained by multiplying the merged motion vector offset precision by a non-zero integer.

6. The encoding device according to claim 4, wherein, Merge candidates generated based on a weighted average of predefined candidate pairs are added to the merge candidate list.

7. An apparatus for transmitting image data, the apparatus comprising: At least one processor is configured to obtain a bitstream for the image, wherein the bitstream is generated based on the following operations: determining whether a merging mode is applied to the current block among a plurality of inter-frame prediction modes, wherein the plurality of inter-frame prediction modes include the merging mode and a motion vector prediction MVP mode; based on the determination that the merging mode is applied to the current block, configuring a merging candidate list based on motion information of spatial and temporal candidates for the current block under the merging mode; determining merging candidates for deriving corrected merging candidates based on merging motion vector offsets, wherein the merging candidate at position 0 of the merging candidate list... The merge candidate is determined as a merge candidate for deriving the corrected merge candidate based on the merged motion vector offset; the motion information of the corrected merge candidate is derived based on the sum of the motion vectors of the determined merge candidate and the merged motion vector offset derived based on the merged motion vector offset accuracy information; a prediction block of the current block is generated based on the motion information of the corrected merge candidate; and video information including merged motion vector offset accuracy information related to whether the merged motion vector offset has fractional pixel accuracy is generated, wherein the merged motion vector offset accuracy information is included in the picture parameter set of the video information; and A transmitter configured to send the data containing the bit stream. Specifically, after configuring the merged candidate list including the spatial candidates and the temporal candidates, merged candidates generated based on the average of predefined candidate pairs are added to the merged candidate list, and The predefined candidate pair is the pair of the candidate at position 0 and the candidate at position 1 in the merged candidate list.

Images