Image decoding apparatus, image encoding apparatus, and transmission apparatus

Abstract

Image decoding apparatus, image encoding apparatus, and transmission apparatus. A method of decoding a video signal and an apparatus therefor are disclosed. Specifically, a method of decoding an image based on an inter prediction mode includes the steps of generating a merge candidate list by using a spatial merge candidate and a temporal merge candidate of a current block, acquiring a merge index indicating a specific merge candidate from the merge candidate list, and generating a prediction block of the current block by using motion information of the specific merge candidate, wherein the step of generating the merge candidate list can include the step of adding a first merge candidate indicating a collocated block specified by motion information of the spatial merge candidate to the merge candidate list when a reference picture of the spatial merge candidate is identical to a predetermined picture.

Description

[0001] This application is a divisional application of the original patent application No. 201980021678.9 (International Application No.: PCT / KR2019 / 008085, Application Date: July 2, 2019, Invention Title: Image Processing Method Based on Inter-Frame Prediction Mode and Apparatus for the Method). Technical Field

[0002] This disclosure relates to methods for processing still or moving images, and more specifically, to methods for encoding / decoding still or moving images based on inter-frame prediction modes and devices supporting such methods. Background Technology

[0003] Compression coding refers to a series of signal processing techniques used to transmit digital information over communication lines or to store information in a form suitable for storage media. Media including pictures, images, audio, etc., can be targets of compression coding, and specifically, compression coding techniques targeting pictures are called video image compression.

[0004] Next-generation video content should feature high spatial resolution, high frame rates, and high-dimensional scene rendering. Processing this content will necessitate a dramatic increase in memory storage, memory access speeds, and processing power.

[0005] Therefore, there is a need to design encoding tools for efficiently processing next-generation video content. Summary of the Invention

[0006] Technical issues

[0007] Embodiments of this disclosure provide a method and apparatus for reducing hardware resource usage when performing inter-frame prediction (or inter-frame forecasting) based on the similarity of temporal motion information.

[0008] The technical objectives of this disclosure are not limited to those mentioned above, and other technical objectives not mentioned above will be clearly understood by those skilled in the art based on the following description.

[0009] Technical solution

[0010] In one aspect of this disclosure, a method for decoding an image based on an inter-frame prediction mode includes the following steps: generating a merge candidate list by using spatial merge candidates and temporal merge candidates for a current block; obtaining a merge index indicating a specific merge candidate in the merge candidate list; and generating a prediction block for the current block by using motion information of the specific merge candidate, wherein the generation of the merge candidate list may include: if the reference image of the spatial merge candidate is the same as a predetermined image, adding a first merge candidate indicating a juxtaposed block specified by the motion information of the spatial merge candidate to the merge candidate list.

[0011] As an implementation, the generation of the merge candidate list may further include determining the search order of spatial merge candidates previously added to the merge candidate list, wherein the spatial merge candidates may be selected from the merge candidate list based on the search order.

[0012] As an implementation method, the search order can be determined based on at least one of the following: the slice type of the current block, a specific list of reference images defined by the high-level syntax, and whether the image sequence number (POC) of all reference images included in the reference image list of the current block is equal to or less than the POC of the current block.

[0013] As an implementation, if the merge index indicates the first merge candidate, the prediction block can be generated by using the motion information of the sub-blocks of the first merge candidate.

[0014] As an implementation, the generation of the merge candidate list may include adding a second merge candidate, which indicates another juxtaposed block specified by scaling the motion information of the spatial merge candidate, to the merge candidate list if the reference image of the spatial merge candidate is different from the predetermined image.

[0015] In another aspect of this disclosure, an apparatus for decoding an image based on an inter-frame prediction mode includes: a merge candidate list generation unit configured to generate a merge candidate list using spatial merge candidates and temporal merge candidates for a current block; a merge index obtaining unit configured to obtain a merge index indicating a specific merge candidate in the merge candidate list; and a prediction block generation unit configured to generate a prediction block for the current block using motion information of the specific merge candidate, wherein the merge candidate list generation unit may be configured to add a first merge candidate indicating a juxtaposed block specified by the motion information of the spatial merge candidate to the merge candidate list if the reference image of the spatial merge candidate is the same as a predetermined image.

[0016] As an implementation, the merge candidate list generation unit can be configured to determine the search order of the spatial merge candidates previously added to the merge candidate list, wherein the spatial merge candidates can be selected from the merge candidate list based on the search order.

[0017] As an implementation method, the search order can be determined based on at least one of the following: the slice type of the current block, a specific list of reference images defined by the high-level syntax, and whether the image sequence number (POC) of all reference images included in the reference image list of the current block is equal to or less than the POC of the current block.

[0018] As an implementation, if the merge candidate index indicates the first merge candidate, the prediction block can be generated by using the motion information of the sub-blocks of the first merge candidate.

[0019] As an implementation, the merge candidate list generation unit is configured to add a second merge candidate that indicates another juxtaposed block specified by scaling the motion information of the spatial merge candidate if the reference image of the spatial merge candidate is different from the predetermined image.

[0020] Beneficial effects

[0021] According to embodiments of this disclosure, by using motion information from already decoded images, the memory cache hit rate can be increased when executing prediction blocks.

[0022] Furthermore, according to embodiments of this disclosure, by using motion information from already decoded images, computation speed can be increased while minimizing memory usage during inter-frame prediction.

[0023] The effects that can be obtained in this disclosure are not limited to those mentioned above, and those skilled in the art will clearly understand other unmentioned effects based on the following description. Attached Figure Description

[0024] To aid in understanding this disclosure, embodiments of the disclosure are provided in the accompanying drawings, which are included as part of the detailed description, and the technical features of the disclosure are described together with the detailed description.

[0025] Figure 1 This is a schematic block diagram of an encoding apparatus for encoding video / image signals as an embodiment of the present disclosure.

[0026] Figure 2 This is a schematic block diagram of a decoding apparatus for performing video / image signal decoding as an embodiment of the present disclosure.

[0027] Figure 3This is a diagram illustrating an example of a multi-type tree structure to which embodiments of the present disclosure can be applied.

[0028] Figure 4 This is a diagram illustrating a signaling mechanism for partitioning information of a quadtree with nested multi-type tree structures, which is an embodiment of the present disclosure that can be applied.

[0029] Figure 5 This diagram illustrates a method for dividing a CTU into multiple CUs based on quadtrees and nested multi-type tree structures, which can be implemented using the present disclosure.

[0030] Figure 6 This is an illustration of a method for limiting ternary tree segmentation as an embodiment of which the present disclosure can be applied.

[0031] Figure 7 This is a diagram illustrating a redundant partitioning pattern that may occur during binary tree partitioning and ternary tree partitioning, as an embodiment of which this disclosure can be applied.

[0032] Figure 8 and Figure 9 This is a diagram illustrating an inter-frame prediction-based video / image coding method according to an embodiment of the present disclosure and an inter-frame prediction unit in a coding apparatus according to an embodiment of the present disclosure.

[0033] Figure 10 and Figure 11 This is a diagram illustrating a video / image decoding method based on inter-frame prediction according to an embodiment of the present disclosure and an inter-frame prediction unit in a decoding apparatus according to an embodiment of the present disclosure.

[0034] Figure 12 This is a diagram used to describe the neighboring blocks used in merge mode or skip mode as an implementation of the present disclosure.

[0035] Figure 13 This is a flowchart illustrating a method for configuring and merging candidate lists according to an embodiment of the present disclosure.

[0036] Figure 14 This is a flowchart illustrating a method for configuring and merging candidate lists according to an embodiment of the present disclosure.

[0037] Figure 15 and Figure 16 This is a diagram illustrating a method for deriving advanced time motion vector prediction (ATMVP) candidates as an implementation of this disclosure.

[0038] Figure 17This is a diagram illustrating a method for deriving advanced time motion vector prediction (ATMVP) candidates as an implementation of this disclosure.

[0039] Figure 18 An example is illustrated of a method for obtaining motion information from a reference image in order to decode a current image, according to an embodiment of the present disclosure.

[0040] Figure 19 An example of the configuration of a reference list as an embodiment of the present disclosure is shown.

[0041] Figure 20 An example of the configuration of reference images for each prediction mode and each prediction direction for neighboring blocks adjacent to the current block is illustrated as an implementation of this disclosure.

[0042] Figure 21 An example of a search order for generating a list of spatial candidates according to an embodiment of the present disclosure is illustrated.

[0043] Figure 22 This is another example illustrating the search order for generating a list of spatial candidates according to an embodiment of the present disclosure.

[0044] Figure 23 Another example of the configuration of a list of reference pictures as an embodiment of the present disclosure is shown.

[0045] Figure 24 Another example is illustrated, which is an implementation of the present disclosure, of the configuration of reference images for each prediction mode and each prediction direction for neighboring blocks adjacent to the current block.

[0046] Figure 25 This is an example of a search order determined based on whether a reference image has a lower Picture Order Number (POC) than the current image, according to an embodiment of the present disclosure, for generating a spatial candidate list.

[0047] Figure 26 and Figure 27 Examples of search orders configured by taking into account priority directions defined in a higher-level syntax, according to embodiments of the present disclosure, are illustrated.

[0048] Figure 28 and Figure 29 An example is shown of a reference image predefined by taking into account the picture sequence number (POC) of the current image and the POC of the reference image as an embodiment of applying this disclosure.

[0049] Figure 30 and Figure 31 Examples of methods for selecting space candidates for ATMVP as an implementation of this disclosure are illustrated.

[0050] Figure 32 An example is illustrated as an embodiment of the present disclosure of a method for deriving a temporal motion vector by scaling a spatial candidate motion vector.

[0051] Figure 33 and Figure 34 An example is given of a method for deriving spatial motion vectors based on backward mapping as an implementation of the present disclosure.

[0052] Figure 35 This is a flowchart of generating a prediction block according to an embodiment of the present disclosure.

[0053] Figure 36 This is a diagram illustrating an inter-frame prediction apparatus according to an embodiment of the present disclosure.

[0054] Figure 37 An example of a video coding system using this disclosure is provided.

[0055] Figure 38 This is a structural diagram of a content streaming system as an embodiment of the present disclosure. Detailed Implementation

[0056] Preferred embodiments of the present disclosure will be described below with reference to the accompanying drawings. The description below, using the drawings, is intended to illustrate exemplary embodiments of the present disclosure and is not intended to describe the only embodiments that can implement the present disclosure. Specific details are included in the following description to provide a complete understanding of the present disclosure. However, it should be understood that those skilled in the art can implement the present disclosure without these specific details.

[0057] In some cases, to prevent the technical concept of this disclosure from being unclear, well-known structures or devices may be omitted, or these structures or devices may be depicted as block diagrams focusing on the core functions of the structures or devices.

[0058] Furthermore, while commonly used terms are selected as much as possible in this disclosure, terms chosen arbitrarily by the applicant are used in certain cases. Since the meaning of such terms will be clearly described in the corresponding section of the specification in such cases, it should be understood that this disclosure is not to be interpreted simply by the terms used only in the specification of this disclosure, but rather that the meaning of the terms should be clarified.

[0059] Specific terminology may be provided for use in the following description to aid in understanding this disclosure. Furthermore, the specific terminology may be modified to other forms within the scope of the technical concept of this disclosure. For example, signals, data, samples, images, frames, blocks, etc., may be appropriately substituted and interpreted in each encoding process.

[0060] In this disclosure, "processing unit" means a unit that performs encoding / decoding processes such as prediction, transformation, and / or quantization. For ease of description, a processing unit may also be referred to as a "processing block" or "block".

[0061] A processing unit can be interpreted as including units for the luminance component and units for the chrominance component. For example, a processing unit can correspond to a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

[0062] Furthermore, the processing unit can be interpreted as a unit for the luma component or a unit for the chroma component. For example, the processing unit may correspond to a coding tree block (CTB), coding block (CB), prediction block (PB), or transform block (TB) for the luma component. Alternatively, the processing unit may correspond to a coding tree block (CTB), coding block (CB), prediction block (PB), or transform block (TB) for the chroma component. Moreover, this disclosure is not limited thereto, and the processing unit can be interpreted to include the meaning of both units for the luma component and units for the chroma component.

[0063] Furthermore, the processing unit is not inherently limited to square blocks, but can be a polygon with three or more vertices.

[0064] Furthermore, in the following text, in this disclosure, a pixel or pixel element is collectively referred to as a sample. Additionally, using a sample can mean using a pixel value or pixel element value.

[0065] Figure 1 This is a schematic block diagram of an encoding apparatus for encoding video / image signals as an embodiment of the present disclosure.

[0066] Reference Figure 1The encoding device 100 can be configured to include an image segmentation unit 110, a subtraction unit 115, a transform unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse transform unit 150, an addition unit 155, a filtering unit 160, a memory 170, an inter-frame prediction unit 180, an intra-frame prediction unit 185, and an entropy coding unit 190. The inter-frame prediction unit 180 and the intra-frame prediction unit 185 can be collectively referred to as prediction units. In other words, prediction units can include inter-frame prediction unit 180 and intra-frame prediction unit 185. The transform unit 120, quantization unit 130, inverse quantization unit 140, and inverse transform unit 150 can be included in a residual processing unit. The residual processing unit can also include a subtraction unit 115. As an implementation, the image segmentation unit 110, subtraction unit 115, transform unit 120, quantization unit 130, inverse quantization unit 140, inverse transform unit 150, addition unit 155, filtering unit 160, inter-frame prediction unit 180, intra-frame prediction unit 185, and entropy coding unit 190 can be configured by a single hardware component (e.g., an encoder or processor). Additionally, the memory 170 may include a decoded image buffer (DPB) and can be configured by a digital storage medium.

[0067] Image segmentation unit 110 divides the input image (or picture or frame) input to encoding device 100 into one or more processing units. As an example, a processing unit may be referred to as a coding unit (CU). In this case, coding units can be recursively divided from coding tree units (CTUs) or maximum coding units (LCUs) according to a quadtree / binary tree structure. For example, a coding unit can be divided into multiple deeper coding units based on a quadtree structure and / or a binary tree structure. In this case, for example, a quadtree structure can be applied first, followed by a binary tree structure. Alternatively, a binary tree structure can be applied first. The encoding process according to this disclosure can be performed based on the final coding units that have not been further divided. In this case, the maximum coding unit can be directly used as the final coding unit based on the encoding efficiency depending on the image characteristics, or the coding unit can be recursively divided into deeper coding units if necessary, resulting in the use of a coding unit with the optimal size as the final coding unit. Here, the encoding process may include processes including prediction, transformation, and reconstruction, as described below. As another example, the processing unit may also include a prediction unit (PU) or a transform unit (TU). In this case, each of the prediction unit and the transform unit can be divided or segmented from the final encoding unit described above. The prediction unit may be a unit for predicting samples, and the transform unit may be a unit for deriving transform coefficients and / or deriving the residual signal using the transform coefficients.

[0068] In some cases, units can be used in combination with terms such as blocks or regions. Typically, an M×N block can refer to a set of samples or transform coefficients consisting of M columns and N rows. Samples can usually indicate pixels or pixel values, pixel / pixel values ​​for only the luminance component, or pixel / pixel values ​​for only the chrominance component. In the case of samples, a picture (or image) can be used as the term corresponding to pixels or picometers.

[0069] Encoding unit 100 subtracts the prediction signal (or prediction block or prediction sample array) output from inter-frame prediction unit 180 or intra-frame prediction unit 185 from the input image signal to generate a residual signal (or residual block or residual sample array), and the generated residual signal is sent to transform unit 120. In this case, as illustrated, the unit in encoding unit 100 that subtracts the prediction signal (prediction block or prediction sample array) from the input image signal (original block or original sample array) can be referred to as subtraction unit 115. The prediction unit can perform prediction on the processing target block (hereinafter referred to as the current block) and generate a prediction block including prediction samples of the current block. The prediction unit can determine whether to apply intra-frame prediction or inter-frame prediction on the basis of the current block or CU. The prediction unit can generate various information about the prediction, such as prediction mode information, and transmit the generated information to entropy coding unit 190, as described below when describing each prediction mode. The information about the prediction can be encoded in entropy coding unit 190 and output in the form of a bitstream.

[0070] Intra-prediction unit 185 can predict the current block by referencing samples in the current image. Depending on the prediction mode, the referenced samples may be near or separate from the current block. In intra-prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC mode and planar mode. Depending on the precision of the prediction direction, the directional modes can include, for example, 33 or 65 directional prediction modes. However, this is just an example, and depending on the configuration, more or fewer directional prediction modes may be used. Intra-prediction unit 185 can determine the prediction mode to be applied to the current block by using prediction modes applied to neighboring blocks.

[0071] The inter-frame prediction unit 180 can deduce the prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference image. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current image and temporally neighboring blocks existing in the reference image. The reference image including the reference block and the reference image including the temporally neighboring block may be the same as or different from each other. The temporally neighboring block may be referred to by names such as juxtaposed reference block, juxtaposed CU (colCU), etc., and the reference image including the temporally neighboring block may be referred to as juxtaposed image (colPic). For example, the inter-frame prediction unit 180 may configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate to use for deduce the motion vector and / or reference image index of the current block. Inter-frame prediction can be performed based on various prediction modes, and for example, in skip mode and merge mode, the inter-frame prediction unit 180 can use motion information of neighboring blocks as motion information of the current block. In skip mode, residual signals can be sent instead of in merge mode. In motion vector prediction (MVP) mode, motion vectors of neighboring blocks are used as motion vector predictors, and motion vector differences are signaled to indicate the motion vector of the current block.

[0072] The prediction signal generated by the inter-frame prediction unit 180 or the intra-frame prediction unit 185 can be used to generate the reconstructed signal or the residual signal.

[0073] Transform unit 120 can generate transform coefficients by applying transform techniques to the residual signal. For example, the transform techniques may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, when the relationship information between pixels is represented by a graph, GBT refers to the transform obtained from that graph. CNT refers to the transform obtained by generating a prediction signal using all previously reconstructed pixels and based on the generated prediction signal. Furthermore, transform processing can be applied to square pixel blocks of the same size or even to non-square blocks of variable size.

[0074] Quantization unit 130 can quantize the transform coefficients and send the quantized transform coefficients to entropy coding unit 190. Entropy coding unit 190 can encode the quantized signal (information about the quantized transform coefficients) and output the encoded quantized signal as a bitstream. The information about the quantized transform coefficients can be referred to as residual information. Quantization unit 130 can rearrange the block-quantized transform coefficients into a one-dimensional vector based on the coefficient scan order, and generate information about the quantized transform coefficients based on the one-dimensional vector quantized transform coefficients. Entropy coding unit 190 can perform various coding methods, including (e.g.) exponential Golomb, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), etc. In addition to the quantized transform coefficients, entropy coding unit 190 can also encode information required for reconstructing the video / image (e.g., values ​​of syntax elements, etc.) together or separately. The encoded information (e.g., encoded video / image information) can be sent or stored in bitstream form at the network abstraction layer (NAL) unit level. The bit stream can be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media, including USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter (not shown) that transmits the signal output from the entropy encoding unit 190 and / or the storage device (not shown) that stores the signal may be configured as an internal / external element of the encoding device 100, or the transmitter may be a component of the entropy encoding unit 190.

[0075] The quantized transform coefficients output from quantization unit 130 can be used to generate a prediction signal. For example, in a loop, the vectorized transform coefficients from dequantization unit 140 and inverse transform unit 150 are dequantized and inverse transformed to reconstruct the residual signal. Addition unit 155 can add the reconstructed residual signal to the prediction signal output from inter-frame prediction unit 180 or intra-frame prediction unit 185 to generate a reconstructed signal (reconstructed image, reconstructed block, or reconstructed sample array). As with the case of applying skip mode, when the target block to be processed has no residual, the prediction block can be used as the reconstructed block. Addition unit 155 can be referred to as a reconstruction unit or a reconstructed block generation unit. The generated reconstructed signal can be used for intra-frame prediction of the next target block to be processed in the current image, and for inter-frame prediction of the next image by filtering as described below.

[0076] The filtering unit 160 can enhance the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filtering unit 160 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in the memory 170, specifically in the DPB of the memory 170. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc. The filtering unit 160 can generate various information about the filtering and transmit the generated information to the entropy coding unit 190, as described below in the description of each filtering method. The information about the filtering can be encoded by the entropy coding unit 190 and output as a bitstream.

[0077] The inter-frame prediction unit 180 can use the modified reconstructed image sent to the memory 170 as a reference image. When applying inter-frame prediction through it, the encoding device can avoid prediction mismatch between the encoding device 100 and the decoding device, and also improves the encoding efficiency.

[0078] The DPB of memory 170 can store modified reconstructed images so that they can be used as reference images in inter-frame prediction unit 180. Memory 170 can store motion information of blocks in which motion information in the current image is derived (or encoded) and / or motion information of blocks in already reconstructed images. The stored motion information can be transmitted to inter-frame prediction unit 180 so that it can be used as motion information for spatially or temporally neighboring blocks. Memory 170 can store reconstructed samples of reconstructed blocks in the current image and transmit the stored reconstructed samples to intra-frame prediction unit 185.

[0079] Figure 2 This is a schematic block diagram of a decoding apparatus for performing video / image signal decoding as an embodiment of the present disclosure.

[0080] Reference Figure 2The decoding device 200 can be configured to include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an addition unit 235, a filtering unit 240, a memory 250, an inter-frame prediction unit 260, and an intra-frame prediction unit 265. The inter-frame prediction unit 260 and the intra-frame prediction unit 265 can be collectively referred to as prediction units. In other words, a prediction unit may include inter-frame prediction unit 260 and intra-frame prediction unit 265. The inverse quantization unit 220 and the inverse transform unit 230 can be collectively referred to as residual processing units. In other words, a residual processing unit may include inverse quantization unit 220 and inverse transform unit 230. According to an embodiment, the entropy decoding unit 210, the inverse quantization unit 220, the inverse transform unit 230, the addition unit 235, the filtering unit 240, the inter-frame prediction unit 260, and the intra-frame prediction unit 265 can be configured by a single hardware component (e.g., a decoder or a processor). Additionally, the memory 170 may include a decoded image buffer (DPB) and can be configured by a digital storage medium.

[0081] When the input includes a bitstream containing video / image information, the decoding device 200 can correspond to... Figure 1 The process of processing video / image information in an encoding apparatus is used to reconstruct the image. For example, the decoding apparatus 200 can use a processing unit applied in the encoding apparatus to perform decoding. Therefore, the decoding processing unit can be, for example, an encoding unit, and the encoding unit can be segmented from the encoding tree unit or the maximum encoding unit according to a quadtree structure and / or a binary tree structure. In addition, the reconstructed image signal decoded and output by the decoding apparatus 200 can be reproduced by a playback apparatus.

[0082] Decoding device 200 can receive data from a source in the form of a bitstream. Figure 1The signal output by the encoding device can be decoded by the entropy decoding unit 210. For example, the entropy decoding unit 210 can deduce the information required for image reconstruction (or picture reconstruction) (e.g., video / image information) by parsing the bitstream. For example, the entropy decoding unit 210 can decode the information in the bitstream based on encoding methods such as exponential Golomb coding, CABAC, or CAVLC, and output the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients for the residuals. More specifically, in the CABAC entropy decoding method, bins corresponding to each syntax element are received in the bitstream, a context model is determined by using decoding information of neighboring blocks and the target block and decoding target syntax element information or information about the symbols / bins decoded in previous steps, and arithmetic decoding of bins is performed by predicting the generation probability of bins based on the determined context model to generate symbols corresponding to the values ​​of each syntax element. In this case, in the CABAC entropy decoding method, after determining the context model, the context model can be updated by using the information of the decoded symbols / bins for the context model of the next symbol / bin. The prediction information in the information decoded by the entropy decoding unit 210 can be provided to the prediction units (inter-frame prediction unit 260 and intra-frame prediction unit 265), and the residual value of the entropy decoding by the entropy decoding unit 210, i.e., the quantized transform coefficients and related parameter information, can be input to the dequantization unit 220. Additionally, the filtering information in the information decoded by the entropy decoding unit 210 can be provided to the filtering unit 240. Furthermore, the receiver (not shown) that receives the signal output from the encoding device can be configured as an internal / external component of the decoding device 200, or the receiver can be a component of the entropy decoding unit 210.

[0083] The dequantization unit 220 dequantizes the quantized transform coefficients to output transform coefficients. The dequantization unit 220 can rearrange the quantized transform coefficients into a 2D block type. In this case, the rearrangement can be performed based on the coefficient scan order executed by the encoding device. The dequantization unit 220 can perform dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain the transform coefficients.

[0084] The inverse transform unit 230 performs an inverse transform on the transform coefficients to obtain the residual signal (residual block or residual sample array).

[0085] The prediction unit can perform prediction on the current block and generate a prediction block that includes prediction samples for the current block. The prediction unit can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the prediction information output from the entropy decoding unit 210, and determine the specific intra-frame / inter-frame prediction mode.

[0086] Intra-prediction unit 265 can predict the current block by referencing samples in the current image. Depending on the prediction mode, the referenced samples may be near or separate from the current block. In intra-prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. Intra-prediction unit 265 can determine the prediction mode applied to the current block by using prediction modes applied to neighboring blocks.

[0087] The inter-frame prediction unit 260 can deduce the prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference image. In this case, to reduce the amount of motion information transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the motion information correlation between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current image and temporally neighboring blocks existing in the reference image. For example, the inter-frame prediction unit 260 can configure a motion information candidate list based on neighboring blocks and deduce the motion vector and / or reference image index of the current block based on the received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the mode of inter-frame prediction for the current block.

[0088] Adding unit 235 adds the obtained residual signal to the prediction signal (prediction block or prediction sample array) output from inter-frame prediction unit 260 or intra-frame prediction unit 265 to generate a reconstruction signal (reconstructed image, reconstruction block, or reconstruction sample array). Similar to the case of applying skip mode, when there is no residual in the target block, the prediction block can be used as the reconstruction block.

[0089] Addition unit 235 can be referred to as reconstruction unit or reconstruction block generation unit. The generated reconstruction signal can be used for intra-frame prediction of the next processing target block in the current image, and for inter-frame prediction of the next image by filtering as described below.

[0090] The filtering unit 240 can enhance the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filtering unit 240 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and send the modified reconstructed image to the memory 250, specifically to the DPB of the memory 250. Various filtering methods may include, for example, deblocking filtering, adaptive sample shifting, adaptive loop filtering, bilateral filtering, etc.

[0091] The (modified) reconstructed image stored as a reference image in the DPB of memory 250 can be used in inter-frame prediction unit 260. Memory 250 can store motion information of blocks in which motion information in the current image is derived (or decoded) and / or motion information of blocks in already reconstructed images. The stored motion information can be transmitted to inter-frame prediction unit 260 so as to be used as motion information of spatially or temporally neighboring blocks. Memory 250 can store reconstructed samples of reconstructed blocks in the current image and transmit the stored reconstructed samples to intra-frame prediction unit 265.

[0092] In this disclosure, the embodiments described in the filtering unit 160, inter-frame prediction unit 180, and intra-frame prediction unit 185 of the encoding device 100 can be applied in a similar or even corresponding manner to the filtering unit 240, inter-frame prediction unit 260, and intra-frame prediction unit 265 of the decoding device 200.

[0093] Block segmentation

[0094] The video / image coding method according to this disclosure can be performed based on various detailed techniques, and the corresponding detailed techniques are illustrated below. It will be apparent to those skilled in the art that the following techniques can be associated with related processes including prediction, residual processing (inverse transform, inverse quantization, etc.), syntax element encoding, filtering, segmentation / partitioning, etc. in the video / image coding / decoding process described above and / or below.

[0095] The image segmentation unit 110 of the aforementioned encoding apparatus performs the block segmentation process according to this disclosure, and the segmentation-related information can be processed (encoded) by the entropy encoding unit 190 and transmitted to the decoding apparatus in the form of a bitstream. The entropy decoding unit 210 of the decoding apparatus can deduce the block segmentation structure of the current image based on the segmentation-related information obtained from the bitstream, and can perform a series of processes for image decoding (e.g., prediction, residual processing, block reconstruction, in-loop filtering, etc.) based on the deduced block segmentation structure.

[0096] Segment the image into CTUs

[0097] An image can be divided into a series of coding tree units (CTUs). Each CTU can correspond to a coding tree block (CTB). Alternatively, a CTU may include a coding tree block for luma samples and two corresponding coding tree blocks for chroma samples. In other words, for an image comprising three sample arrays, a CTU may include an N×N block of luma samples and two corresponding blocks of chroma samples.

[0098] The maximum support size of the CTU used for encoding and prediction can differ from the maximum support size of the CTU used for transformation. For example, the maximum support size of the luma block in the CTU can be 128×128.

[0099] CTU segmentation using a tree structure

[0100] The CTU can be segmented into CUs based on a quadtree (QT) structure. The quadtree structure can be referred to as a quaternary tree structure. This is to reflect various local characteristics. Furthermore, in this disclosure, in addition to quadtrees, the CTU can also be segmented based on multiple types of tree structures, including binary trees (BT) and ternary trees (TT). Hereinafter, a QTBT structure can include segmentation structures based on quadtrees and binary trees, and QTBTTT can include segmentation structures based on quadtrees, binary trees, and ternary trees. Alternatively, a QTBT structure can include segmentation structures based on quadtrees, binary trees, and ternary trees. In the coding tree structure, the CU can have a square or rectangular shape. The CTU can first be segmented into a quadtree structure. Subsequently, the leaf nodes of the quadtree structure can be further segmented according to multiple types of tree structures.

[0101] Figure 3 This is a diagram illustrating various types of tree structures that can be applied to this disclosure.

[0102] In embodiments of this disclosure, the multi-type tree structure may include Figure 3 The four segmentation types are illustrated below. These four segmentation types can include vertical binary segmentation (SPLIT_BT_VER), horizontal binary segmentation (SPLIT_BT_HOR), vertical ternary segmentation (SPLIT_TT_VER), and horizontal ternary segmentation (SPLIT_TT_HOR). Leaf nodes in the multi-type tree structure can be referred to as CUs. CUs can be used in prediction and transformation processes. In this disclosure, typically, CUs, PUs, and TUs can have the same block size. However, when the maximum supported transformation length is less than the width or height of the color component of the CU, the CUs and TUs can have different block sizes.

[0103] Figure 4 This is a diagram illustrating a signaling mechanism for partitioning information of a quadtree with nested multi-type tree structures, which is an embodiment of the present disclosure that can be applied.

[0104] Here, the CTU is treated as the root of a quadtree and is first split into a quadtree structure. Subsequently, each quadtree leaf node can be further split into multi-type tree structures. In the multi-type tree structure, a first flag (e.g., `mtt_split_cu_flag`) is signaled to indicate whether the corresponding node should be further split. When the corresponding node is further split, a second flag (e.g., `mtt_split_cu_vertical_flag`) can be signaled to indicate the split direction. Subsequently, a third flag (e.g., `mtt_split_cu_binary_flag`) can be signaled to indicate whether the split type is a binary split or a ternary split. For example, based on `mtt_split_cu_vertical_flag` and `mtt_split_cu_binary_flag`, the multi-type tree splitting pattern of the CU can be derived as shown in Table 1 below.

[0105] [Table 1]

[0106] MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1

[0107] Figure 5 This diagram illustrates a method for dividing a CTU into multiple CUs based on quadtrees and nested multi-type tree structures, which can be implemented using the present disclosure.

[0108] Here, the bolded block edges indicate quadtree partitioning, while the remaining edges indicate multi-type tree partitioning. Quadtree partitioning accompanying multi-type trees can provide a content-adaptive coding tree structure. A CU can correspond to a coding block (CB). Alternatively, a CU can include a coding block for luma samples and two coding blocks for corresponding chroma samples. The size of a CU can be the same as a CTU, or it can be as small as 4×4 with luma samples as units. For example, in the case of a 4:2:0 color format (or chroma format), the maximum chroma CB size can be 64×64, and the minimum chroma CB size can be 2×2.

[0109] In this disclosure, for example, the maximum supported luminance TB size can be 64×64, and the maximum supported chrominance TB size can be 32×32. When the width or height of the CB segmented according to the tree structure is greater than the maximum transform width or height, the corresponding CB can be automatically (or implicitly) segmented until the horizontal and vertical TB size limits are met.

[0110] Furthermore, for quadtree encoding tree schemes that accompany multiple tree types, the following parameters can be defined and identified as SPS syntax elements.

[0111] -CTU size: Size of the root node of the quadtree

[0112] -MinQTSize: Minimum allowed size of quadtree leaf nodes

[0113] -MaxBtSize: Maximum allowed size of the root node of the binary tree

[0114] -MaxTtSize: Maximum allowed size of the root node of the ternary tree

[0115] -MaxMttDepth: The maximum allowed depth of hierarchical division of a multi-type tree from quadtree leaves.

[0116] -MinBtSize: Minimum allowed size of binary leaf nodes

[0117] -MinTtSize: Minimum allowed size of a ternary leaf node

[0118] As an example of a quadtree coding tree structure accompanying a multi-type tree, the CTU size can be configured as a 64×64 block (in 4:2:0 chroma format) with 128×128 luma samples and two corresponding chroma samples. In this case, MinOTSize can be set to 16×16, MaxBtSize can be set to 128×128, MaxTtSize can be set to 64×64, and MinBtSize and MinTtSize (for both width and height) can be set to 4×4, and MaxMttDepth can be set to 4. Quadtree partitioning can be applied to the CTU, and this quadtree partitioning can generate quadtree leaf nodes. Quadtree leaf nodes can be called leaf QT nodes. Quadtree leaf nodes can have sizes from 16×16 (i.e., MinOTSize) to 128×128 (i.e., CTU SIZE). When the leaf QT node is 128×128, the leaf QT node cannot be partitioned into a binary / ternary tree. The reason is that in this case, even if the leaf QT node is split, its size exceeds MaxBtsize and MaxTtsize (i.e., 64×64). In other cases, the leaf QT node can be further split into a multi-type tree. Therefore, the leaf QT node can be the root node for the multi-type tree, and the multi-type tree depth (mttDepth) value of the leaf QT node can be 0. When the multi-type tree depth reaches MaxMttdepth (e.g., 4), additional splits are no longer considered. When the width of the multi-type tree node is equal to MinBtSize and equal to or less than 2×MinTtSize, additional horizontal splits are no longer considered. When the height of the multi-type tree node is equal to MinBtSize and less than or equal to 2×MinTtSize, additional vertical splits are no longer considered.

[0119] Figure 6This is an illustration of a method for limiting ternary tree segmentation as an embodiment of which the present disclosure can be applied.

[0120] Reference Figure 6 To allow for 64×64 luma blocks and 32×32 chroma pipeline designs in hardware decoders, TT segmentation can be restricted under certain conditions. For example, TT segmentation can be restricted when the width or height of the luma encoding block exceeds a predetermined specific value (e.g., 32 or 64). Figure 6 exemplified in .

[0121] In this disclosure, the coding tree scheme supports luma blocks and chroma blocks having separate block tree structures. Regarding P-slices and B-slices, the luma and chroma CTBs within a CTU can be restricted to having the same coding tree structure. However, regarding I-slices, luma blocks and chroma blocks can have separate block tree structures relative to each other. When applying a separate block tree mode, the luma CTB can be segmented into CUs based on a specific coding tree structure, and the chroma CTB can be segmented into chroma CUs based on another coding tree structure. This may mean that a CU in an I-slice can consist of a coding block for the luma component and two coding blocks for the chroma components, and a CU in a P or B-slice can consist of blocks for three color components.

[0122] The section "Splitting CTUs Using Tree Structures" describes the quadtree-encoded tree structure that accompanies multiple tree types; however, the splitting structure of a CU is not limited to this. For example, BT and TT structures can be interpreted as concepts included in a multi-split tree (MPT) structure, and a CU can be interpreted as being split via QT and MPT structures. In the example of splitting a CU via QT and MPT structures, a syntax element (e.g., MPT_split_type) that includes information about how many blocks the leaf nodes of the QT structure are split into, and a syntax element (e.g., MPT_split_mode) that includes information about which direction (vertical or horizontal) the leaf nodes of the QT structure are split into, are signaled to determine the splitting structure.

[0123] In another example, the CU can be segmented using a method different from the QT, BT, or TT structures. In other words, unlike the cases where a lower-depth CU is segmented into 1 / 4 the size of a higher-depth CU according to the QT structure, into 1 / 2 the size of a higher-depth CU according to the BT structure, or into 1 / 4 or 1 / 2 the size of a higher-depth CU according to the TT structure, a lower-depth CU can be segmented into 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 the size of a higher-depth CU, and the segmentation method of the CU is not limited to these.

[0124] If a portion of a tree node block extends beyond the bottom or right image boundary, the corresponding tree node block can be restricted so that all samples of all encoded CUs remain within the image boundary. In this case, for example, the following segmentation rule can be applied.

[0125] -If part of a tree node block extends beyond both the bottom and right image boundaries.

[0126] - If the block is a QT node and the block size is greater than the minimum QT size, then the block is forced to be split in QT splitting mode.

[0127] Otherwise, force the blocks to be split in the SPLIT_BT_HOR mode.

[0128] Otherwise, if part of a tree node block extends beyond the bottom image boundary,

[0129] - If the block is a QT node and the block size is greater than the minimum QT size and the block size is greater than the maximum BT size, then the block is forced to be split in QT splitting mode.

[0130] Otherwise, if the block is a QT node and the block size is greater than the minimum QT size and the block size is less than or equal to the maximum BT size, then the block is forced to be split in either QT split mode or SPLIT_BT_HOR mode.

[0131] Otherwise (if the block is a BTT node or the block size is less than or equal to the minimum QT size), the block will be forced to be split in the SPLIT_BT_HOR mode.

[0132] Otherwise, if part of the tree node block extends beyond the right image boundary,

[0133] - If the block is a QT node and the block size is greater than the minimum QT size and the block size is greater than the maximum BT size, then the block is forced to be split in QT splitting mode.

[0134] Otherwise, if the block is a QT node and the block size is greater than the minimum QT size and the block size is less than or equal to the maximum BT size, then the block is forced to be split in QT split mode or SPLIT_BT_VER mode.

[0135] Otherwise (if the block is a BTT node and the block size is less than or equal to the minimum QT size), the block is forced to be split in SPLIT_BT_VER mode.

[0136] The quadtree coding block structure accompanying multi-type trees can provide a very flexible block partitioning structure. Due to the partitioning types supported by multi-type trees, different partitioning patterns may result in the same coding block structure in certain situations. The generation of redundant partitioning patterns is limited to reduce the amount of data involved in the partitioning information. Redundant partitioning patterns will be described with reference to the following figures.

[0137] Figure 7 This is a diagram illustrating a redundant partitioning pattern that may occur during binary tree partitioning and ternary tree partitioning, as an embodiment of which this disclosure can be applied.

[0138] like Figure 7 As illustrated, a two-level consecutive binary segmentation in one direction has the same coded block structure as the binary segmentation of the center segment following a ternary segmentation. In this case, the binary segmentation (in a given direction) of the center segmentation of the ternary segmentation can be restricted. This restriction can be applied to the CUs of all images. When a particular segmentation is restricted, the signaling of the syntax elements can be modified to reflect this restriction, and the number of bits signaled for segmentation can be reduced through the modified signaling. For example, as... Figure 7 In the example shown, when the binary tree split of the CU's center split is restricted, the syntax element mtt_split_cu_binary_flag indicating whether the split is a binary or ternary split can be left unsigned, and this value can be inferred as 0 by the decoder.

[0139] predict

[0140] To reconstruct the current processing unit in which decoding is performed, the current image or the decoded portion of another image that includes the current processing unit can be used.

[0141] An image that is used only for reconstruction (i.e., to perform intra-frame prediction) can be called an intra-frame image or I-image (slice). An image (slice) that uses up to one motion vector and reference index to predict each cell can be called a prediction image or P-image (slice). An image (slice) that uses up to two motion vectors and reference indices can be called a bidirectional prediction image or B-image (slice).

[0142] Intra-frame prediction refers to a method of deriving predictions for the current processing block from data elements (e.g., sample values) of the same decoded image (or slice). In other words, intra-frame prediction means predicting the pixel values ​​of the current processing block by referring to the reconstructed regions in the current image.

[0143] Inter-frame prediction will be described in more detail below.

[0144] Inter-frame prediction (or inter-image prediction)

[0145] Inter-frame prediction refers to a method of deriving predictions for the current processing block based on data elements (e.g., sample values ​​or motion vectors) from images other than the current image. In other words, inter-frame prediction means predicting the pixel values ​​of the current processing block by referring to reconstructed regions in other reconstructed images besides the current image.

[0146] Inter-frame prediction (or inter-image prediction), a technique for eliminating redundancy between images, is primarily performed through motion estimation and motion compensation.

[0147] In this disclosure, the above Figure 1 and Figure 2 The inter-frame prediction method described in the document is described in detail, and the decoder can be represented as described below. Figure 10 Video / image decoding methods based on inter-frame prediction and Figure 11 The inter-frame prediction unit in the decoding device. Furthermore, the encoder can be represented as described below. Figure 8 Video / image coding methods based on inter-frame prediction and Figure 9 The inter-frame prediction unit in the coding device. Additionally, through... Figure 8 and Figure 9 Encoded data can be stored as a bit stream.

[0148] The prediction unit of the encoding / decoding apparatus can derive prediction samples by performing inter-frame prediction on a block-by-block basis. Inter-frame prediction can be represented as a prediction derived in a way that depends on data elements (e.g., sample values ​​or motion information) of images other than the current image. When applying inter-frame prediction to the current block, the prediction block (prediction sample array) of the current block can be derived based on the reference block (reference sample array) specified by the motion vector on the reference image indicated by the reference image index.

[0149] In this scenario, to reduce the amount of motion information transmitted in inter-frame prediction mode, the motion information of the current block can be predicted on a block, sub-block, or sample basis based on the correlation between motion information between neighboring blocks and the current block. Motion information can include motion vectors and reference image indices. It can also include inter-frame prediction type information (L0 prediction, L1 prediction, Bi prediction, etc.).

[0150] When applying inter-frame prediction, neighboring blocks can include spatially neighboring blocks present in the current image and temporally neighboring blocks present in a reference image. The reference image including the reference block and the reference image including the temporally neighboring block can be the same as or different from each other. The temporally neighboring block can be referred to by names such as juxtaposed reference block, juxtaposed CU (colCU), etc., and the reference image including the temporally neighboring block can be referred to as the juxtaposed image (colPic). For example, a motion information candidate list can be configured based on the neighboring blocks of the current block, and a signal can be sent indicating which candidate's flag or index information to select (use) in order to derive the motion vector of the current block and / or the reference image index.

[0151] Inter-frame prediction can be performed based on various prediction modes, and for example, in skip mode and merge mode, the motion information of the current block can be the same as the motion information of the selected neighboring blocks. In skip mode, residual signals can be sent instead of in merge mode. In motion vector prediction (MVP) mode, the motion vectors of the selected neighboring blocks can be used as motion vector predictors, and the motion vector difference can be signaled. In this case, the motion vector of the current block can be derived using the sum of the motion vector predictors and the motion vector difference.

[0152] Figure 8 and Figure 9 This is a diagram illustrating an inter-frame prediction-based video / image coding method according to an embodiment of the present disclosure and an inter-frame prediction unit in a coding apparatus according to an embodiment of the present disclosure.

[0153] Reference Figure 8 and Figure 9 S801 can be executed by the inter-frame prediction unit 180 of the coding apparatus, and S802 can be executed by the residual processing unit of the coding apparatus. Specifically, S802 can be executed by the subtraction unit 115 of the coding apparatus. In S803, the prediction information can be derived by the inter-frame prediction unit 180 and encoded by the entropy coding unit 190. In S803, the residual information can be derived by the residual processing unit and encoded by the entropy coding unit 190. The residual information is information about the residual samples. The residual information may include information about the quantization transform coefficients for the residual samples.

[0154] As described above, the residual samples can be derived into transform coefficients by the transform unit 120 of the encoding device, and the transform coefficients can be derived into quantized transform coefficients by the quantization unit 130. The information about the quantized transform coefficients can be encoded by the entropy encoding unit 190 using the residual encoding process.

[0155] The encoding device performs inter-frame prediction on the current block (S801). The encoding device can deduce the inter-frame prediction mode and motion information of the current block, and generate prediction samples for the current block. Here, the inter-frame prediction mode determination process, the motion information derivation process, and the prediction sample generation process can be executed simultaneously, and any one of these processes can be executed earlier than the other processes. For example, the inter-frame prediction unit 180 of the encoding device may include a prediction mode determination unit 181, a motion information derivation unit 182, and a prediction sample derivation unit 183. The prediction mode determination unit 181 can determine the prediction mode of the current block, the motion information derivation unit 182 can deduce the motion information of the current block, and the prediction sample derivation unit 183 can deduce the motion samples of the current block.

[0156] For example, the inter-frame prediction unit 180 of the coding apparatus can search for blocks similar to the current block in a predetermined region (search region) of the reference image through motion estimation, and deduce a reference block whose difference from the current block is minimal, equal to, or less than a predetermined standard. Based on this, a reference image index indicating the reference block's location can be derived, and motion vectors can be derived based on the positional difference between the reference block and the current block. The coding apparatus can determine the prediction mode applicable to the current block among various prediction modes. The coding apparatus can compare the RD costs of various prediction modes and determine the optimal prediction mode for the current block.

[0157] For example, when applying a skip mode or merge mode to the current block, the encoding device can configure a merge candidate list, as described below, and deduce a reference block among the reference blocks indicated by the merge candidates included in the merge candidate list whose difference from the current block is minimal, equal to, or less than a predetermined standard. In this case, a merge candidate associated with the deduced reference block can be selected, and merge index information indicating the selected merge candidate can be generated and signaled to the decoding device. The motion information of the current block can be deduced using the motion information of the selected merge candidate.

[0158] As another example, when applying the (A)MVP mode to the current block, the encoding device can configure an (A)MVP candidate list as described below, and use the motion vector of the MVP candidate selected from the motion vector prediction sub-MVP candidates included in the (A)MVP candidate list as the MVP of the current block. In this case, for example, the motion vector of the reference block derived by motion estimation can be used as the motion vector of the current block, and the MVP candidate with the motion vector having the smallest difference from the motion vector of the current block can be selected as the MVP candidate. The motion vector difference (MVD) can be derived as the difference obtained by subtracting the MVP from the motion vector of the current block. In this case, information about the MVD can be signaled to the decoding device. In addition, when applying the (A)MVP mode, the value of the reference image index can be configured as reference image index information and signaled separately to the decoding device.

[0159] The encoding device can derive residual samples based on the predicted samples (S802). The encoding device can derive residual samples by comparing the original samples of the current block with the predicted samples.

[0160] The encoding device encodes image information including prediction information and residual information (S803). The encoding device can output the encoded image information in the form of a bitstream. The prediction information may include information about prediction mode information (e.g., skip flag, merge flag, or mode index, etc.) and information about motion information as information related to the prediction process. The information about motion information may include candidate selection information (e.g., merge index, MVP flag, or MVP index) as information used to derive motion vectors. In addition, the information about motion information may include information about MVD and / or reference image index information.

[0161] Additionally, motion information may include indications of whether L0, L1, or bidirectional prediction was applied. Residual information is about the residual samples. Residual information may include information about the quantization transformation coefficients applied to the residual samples.

[0162] The output bitstream can be stored in (digital) storage media and transmitted to the decoding device, or transmitted to the decoding device via a network.

[0163] Furthermore, as mentioned above, the encoding device can generate a reconstructed image (including reconstructed samples and reconstructed blocks) based on reference samples and residual samples. This is to derive the same prediction result as that performed by the decoding device, thus improving encoding efficiency. Therefore, the encoding device can store the reconstructed image (or reconstructed samples or reconstructed blocks) in memory and use the reconstructed image as a reference image. In-loop filtering can also be applied to the reconstructed image as described above.

[0164] Figure 10 and Figure 11 This is a diagram illustrating a video / image decoding method based on inter-frame prediction according to an embodiment of the present disclosure and an inter-frame prediction unit in a decoding apparatus according to an embodiment of the present disclosure.

[0165] Reference Figure 10 and Figure 11 The decoding device can perform operations corresponding to those performed by the encoding device. The decoding device can perform predictions on the current block based on the received prediction information and derive prediction samples.

[0166] S1001 to S1003 can be executed by the inter-frame prediction unit 260 of the decoding device, and the residual information of S1004 can be obtained from the bitstream by the entropy decoding unit 210 of the decoding device. The residual processing unit of the decoding device can derive the residual samples of the current block based on the residual information. Specifically, the inverse quantization unit 220 of the residual processing unit can derive the transform coefficients by performing inverse quantization based on the quantization transform coefficients derived based on the residual information, and the inverse transform unit 230 of the residual processing unit can derive the residual samples of the current block by performing an inverse transform on the transform coefficients. S1005 can be executed by the addition unit 235 or the reconstruction unit of the decoding device.

[0167] Specifically, the decoding device can determine the prediction mode of the current block based on the received prediction information (S1001). The decoding device can determine which inter-frame prediction mode to apply to the current block based on the prediction mode information in the prediction information.

[0168] For example, the merge flag can be used to determine whether to apply a merge mode or (A)MVP mode to the current block. Alternatively, a variety of inter-frame prediction mode candidates can be selected based on a mode index. Inter-frame prediction mode candidates may include skip mode, merge mode, and / or (A)MVP mode, or may include various inter-frame prediction modes described below.

[0169] The decoding device derives motion information for the current block based on the determined inter-frame prediction mode (S1002). For example, when applying a skip mode or a merge mode to the current block, the decoding device can configure a merge candidate list, as described below, and select one of the merge candidates included in the merge candidate list. The selection can be performed based on selection information (merge index). The motion information for the current block can be derived using the motion information of the selected merge candidate. The motion information of the selected merge candidate can be used as the motion information for the current block.

[0170] As another example, when applying the (A)MVP mode to the current block, the decoding device can configure an (A)MVP candidate list as described below, and use the motion vector of the selected MVP candidate from the motion vector prediction sub-MVP candidates included in the (A)MVP candidate list as the MVP of the current block. Selection can be performed based on selection information (MVP flag or MVP index). In this case, the MVD of the current block can be derived based on information about the MVD, and the motion vector of the current block can be derived based on the MVP and MVD of the current block. Additionally, the reference image index of the current block can be derived based on reference image index information. The image indicated by the reference image index in the reference image list of the current block can be derived as the reference image referenced by the inter-frame prediction of the current block.

[0171] Furthermore, motion information for the current block can be derived without the candidate list configuration described below, and in this case, the motion information for the current block can be derived according to the process disclosed in the prediction mode described below. In this case, the candidate list configuration can be omitted.

[0172] The decoding device can generate a prediction sample for the current block based on the motion information of the current block (S1003). In this case, a reference image can be derived based on the reference image index of the current block, and the prediction sample for the current block can be derived by using the sample of the reference block indicated by the motion vector of the current block on the reference image. In this case, as described below, in some cases, a filtering process for all or some of the prediction samples in the prediction sample of the current block can also be performed.

[0173] For example, the inter-frame prediction unit 260 of the decoding device may include a prediction mode determination unit 261, a motion information derivation unit 262, and a prediction sample derivation unit 263. The prediction mode determination unit 261 may determine the prediction mode of the current block based on the received prediction mode information, the motion information derivation unit 262 may derive the motion information (motion vector and / or reference image index) of the current block based on information about the received motion information, and the prediction sample derivation unit 263 may derive the prediction sample of the current block.

[0174] The decoding device generates residual samples for the current block based on the received residual information (S1004). The decoding device can generate reconstructed samples for the current block based on the predicted samples and residual samples, and generate a reconstructed image based on the generated reconstructed samples (S1005). Thereafter, an in-loop filtering process can be applied to the reconstructed image as described above.

[0175] As described above, the inter-frame prediction process may include an inter-frame prediction mode determination step, a motion information derivation step depending on the determined prediction mode, and a prediction execution (prediction sample generation) step based on the derivation of motion information.

[0176] Determine inter-frame prediction mode

[0177] Various inter-frame prediction modes can be used to predict the current block in an image. For example, various modes can be used, including merge mode, skip mode, MVP mode, affine mode, etc. Decoder-side motion vector correction (DMVR) mode, adaptive motion vector resolution (AMVR) mode, etc., can also be used as auxiliary modes. Affine mode can be called affine motion prediction mode. MVP mode can be called advanced motion vector prediction (AMVP) mode.

[0178] The encoding device can signal the decoding device with prediction mode information indicating the inter-frame prediction mode of the current block. The prediction mode information can be included in the bitstream and received by the decoding device. The prediction mode information may include index information indicating one of several candidate modes. Alternatively, the inter-frame prediction mode can be indicated by hierarchical signaling with flag information. In this case, the prediction mode information may include one or more flags.

[0179] For example, a skip flag can be signaled to indicate whether to apply the skip mode; a merge flag can be signaled to indicate whether to apply the merge mode when the skip mode is not applied; and an MVP mode can be signaled when the merge mode is not applied. Alternatively, a flag for additional differentiation can also be signaled. An affine mode can be signaled as an independent mode or as a subordinate mode to a merge mode or MVP mode. For example, an affine mode can be configured as a candidate in a merge candidate list or an MVP candidate list as described below.

[0180] Motion information is derived from inter-frame prediction modes.

[0181] Inter-frame prediction can be performed using motion information from the current block. The encoding device can derive the optimal motion information for the current block through a motion estimation process. For example, the encoding device can search for highly correlated similar reference blocks in a predetermined search range in a reference image, using original blocks in the original image of the current block, on a fractional pixel basis, and derive motion information from the searched reference blocks. Block similarity can be derived based on the difference between phase-based sample values. For example, block similarity can be calculated based on the SAD between the current block (or its template) and a reference block (or its template). In this case, motion information can be derived based on the reference block with the minimum SAD in the search region. The derived motion information can be signaled to the decoding device based on the inter-frame prediction mode using various methods.

[0182] Merge mode and skip mode

[0183] Figure 12 This is a diagram used to describe the neighboring blocks used in merge mode or skip mode as an implementation of the present disclosure.

[0184] When the merge mode is applied, the motion information of the current prediction block is not sent directly, but is inferred by using the motion information of neighboring prediction blocks. Therefore, a flag indicating the use of the merge mode and a merge index indicating which neighboring prediction block was used are sent to indicate the motion information of the current prediction block.

[0185] The encoder can search for candidate blocks to merge, which are used to derive motion information for the current predicted block, in order to execute a merging pattern. For example, up to five candidate blocks can be used, but this disclosure is not limited thereto. Additionally, a maximum number of candidate blocks can be transmitted in the slice header (or tile group header), and this disclosure is not limited thereto. After finding candidate blocks, the encoder can generate a list of candidate blocks and select the candidate block with the lowest cost from among them as the final candidate block.

[0186] This disclosure provides various implementations for merging candidate blocks that constitute a merge candidate list.

[0187] As a list of merge candidates, for example, five merge candidate blocks could be used. For instance, four spatial merge candidates and one temporal merge candidate could be used. As a concrete example, in the case of spatial merge candidates, Figure 12 The blocks illustrated in the example can be used as space merge candidates.

[0188] Figure 13 This is a flowchart illustrating a method for configuring and merging candidate lists according to an embodiment of the present disclosure.

[0189] Reference Figure 13 , an encoding apparatus (encoder / decoder) inserts a spatial merge candidate derived by searching spatial neighboring blocks of a current block into a merge candidate list (S1301). For example, the spatial neighboring blocks may include a bottom-left neighboring block, a left neighboring block, a top-right neighboring block, a top neighboring block, and a top-left neighboring block of the current block. However, this is an example, and additional neighboring blocks including a right neighboring block, a bottom neighboring block, a bottom-right neighboring block, and the like may be used as the spatial neighboring blocks in addition to the spatial neighboring blocks. The encoding apparatus may derive an available block by searching the spatial neighboring blocks based on priority, and derive motion information of the detected block as a spatial merge candidate. For example, the encoder and the decoder may search in the order of A1, B1, B0, A0, and B2 Figure 12 the five blocks illustrated in, and sequentially index available candidates and configure the indexed candidates as a merge candidate list.

[0190] The encoding apparatus inserts a temporal merge candidate derived by searching temporal neighboring blocks of the current block into the merge candidate list (S1302). The temporal neighboring blocks may be on a reference picture, that is, on a picture different from the current picture where the current block is located. The reference picture where the temporal neighboring blocks are located may be referred to as a collocated picture or col picture. The temporal neighboring blocks may be searched in the order of a bottom-right neighboring block and a bottom-right center block of a collocated block of the current block on the col picture.

[0191] Furthermore, when motion data compression is applied, for each predetermined storage unit, specific motion information may be stored as representative motion information in the col picture. In this case, it is not necessary to store motion information of all blocks in the predetermined storage unit, and as a result, a motion data compression effect can be obtained. In this case, the predetermined storage unit may be predetermined for each 16×16 sample unit or 8×8 sample unit, or size information of the predetermined storage unit may be signaled from the encoder to the decoder. When motion data compression is applied, motion information of a temporal neighboring block may be replaced with representative motion information of a predetermined storage unit where the temporal neighboring block is located.

[0192] In other words, in this case, in terms of implementation, the temporal merge candidate may be derived based on motion information of a prediction block covering a position obtained by performing an arithmetic right shift by a predetermined value and then an arithmetic left shift on the coordinates (top-left sample position) of the temporal neighboring block (instead of the prediction block at the coordinates of the temporal neighboring block). For example, when the predetermined storage unit is a 2n×2n sample unit, if the coordinates of the temporal neighboring block are (xTnb, yTnb), motion information of a prediction block at a modified position of ((xTnb>>n)<<n), (yTnb>>n)<<n)) may be used for the temporal merge candidate.

[0193] Specifically, for example, when the predetermined storage unit is a 16×16 sample unit, if the coordinates of the temporally neighboring block are (xTnb, yTnb), then the motion information of the predicted block at the modified position ((xTnb>>4)<<4), (yTnb>>4)<<4) can be used as a temporal merging candidate. Alternatively, for example, when the predetermined storage unit is an 8×8 sample unit, if the coordinates of the temporally neighboring block are (xTnb, yTnb), then the motion information of the predicted block at the modified position (xTnb>>3)<<3), (yTnb>>3)<<3) can be used as a temporal merging candidate.

[0194] The encoding device can check whether the current number of merging candidates is less than the maximum number of merging candidates (S1303). The maximum number of merging candidates can be predefined or signaled to the decoder from the encoder. For example, the encoder can generate information about the maximum number of merging candidates, encode the generated information, and transmit the encoded information to the decoder in the form of a bit stream. When the maximum number of merging candidates is full, the subsequent candidate addition process may not be performed.

[0195] As a result of the check, when the current number of merge candidates is less than the maximum number of merge candidates, the encoding device inserts additional merge candidates into the merge candidate list (S1304). For example, additional merge candidates may include ATMVP, combined bidirectional prediction merge candidates (when the slice type of the current slice is type B), and / or zero vector merge candidates.

[0196] As a result of the check, if the current number of merge candidates is not less than the maximum number of merge candidates, the encoding device can terminate the configuration of the merge candidate list. In this case, the encoder can select the best merge candidate from the merge candidates constituting the merge candidate list based on the rate distortion (RD) cost, and can signal selection information (e.g., merge index) indicating the selected merge candidate to the decoder. The decoder can select the best merge candidate based on the merge candidate list and the selection information.

[0197] The motion information of the selected merging candidates can be used as the motion information of the current block, and the predicted sample of the current block can be derived based on the motion information of the current block, as described above. The encoder can derive the residual sample of the current block based on the predicted sample and signal the residual information of the residual sample to the decoder. The decoder can generate reconstructed samples based on the residual samples derived based on the residual information and the predicted samples, and generate a reconstructed image based on the reconstructed samples generated as described above.

[0198] When the skip mode is applied, the motion information of the current block can be derived using the same method as when the merge mode is applied. However, when the skip mode is applied, the residual signal of the corresponding block is omitted, and as a result, the predicted sample can be directly used as the reconstructed sample.

[0199] MVP mode

[0200] Figure 14 This is a flowchart illustrating a method for configuring and merging candidate lists according to an embodiment of the present disclosure.

[0201] When applying the Motion Vector Prediction (MVP) pattern, this can be achieved by using reconstructed spatial neighbor blocks (e.g., the above). Figure 12 The motion vectors of the spatially neighboring blocks (as described in the diagram) and / or the motion vectors corresponding to the temporally neighboring blocks (or Col blocks) are used to generate a list of motion vector predictor (MVP) candidates. In other words, the motion vectors of the reconstructed spatially neighboring blocks and / or the motion vectors corresponding to the temporally neighboring blocks can be used as motion vector predictor candidates.

[0202] Information about the prediction can include selection information (e.g., an MVP flag or MVP index) indicating the best motion vector prediction candidate to be selected from the list of motion vector prediction candidates. In this case, the prediction unit can select the motion vector predictor for the current block from the motion vector prediction candidates included in the motion vector candidate list using the selection information. The prediction unit of the encoding device can obtain the motion vector difference (MVD) between the motion vector of the current block and the motion vector predictor, and encode the obtained MVD and output the encoded MVD as a bitstream. In other words, the MVD can be obtained by subtracting the motion vector predictor from the motion vector of the current block. In this case, the prediction unit of the decoding device can obtain the motion vector difference included in the information about the prediction, and derive the motion vector of the current block by adding the motion vector difference to the motion vector predictor. The prediction unit of the decoding device can obtain or derive the reference image index indicating the reference image from the information about the prediction. For example, it can be as follows: Figure 14 The example demonstrates the configuration of the motion vector prediction sub-candidate list.

[0203] Advanced Time Motion Vector Prediction (ATMVP)

[0204] Figure 15 and Figure 16 This is a diagram illustrating a method for deriving advanced time motion vector prediction (ATMVP) candidates as an implementation of this disclosure.

[0205] Reference Figure 15ATMVP is a method that derives the motion information of sub-blocks of a coding unit based on the motion information of juxtaposed blocks of temporally neighboring images. Therefore, it can enhance the performance of Temporal Motion Vector Prediction (TMVP) and reduce the complexity in the general or worst-case scenarios. In this disclosure, ATMVP can be referred to as the Sub-Block-Based Temporal Merging Candidate SbTMVP.

[0206] According to embodiments of this disclosure, the ATMVP can be derived using the following process.

[0207] First, when a neighboring coding unit is available and its motion vector differs from the motion vector in the current candidate list, the encoder / decoder can add a motion vector from the spatially neighboring coding unit. As an example, refer to... Figure 16 This process can be performed in the order of A1, B1, B0, A0, and B2. As another example, to improve complexity, ATMVP can be derived in this process using only the motion vectors of the blocks at fixed positions (e.g., position A1).

[0208] The encoder / decoder can use the first motion vector candidate from N0 available spatial candidates to determine the location for deriving motion information for the juxtaposed image and each sub-block. Here, N0 represents the number of available spatial candidates. If N0 is 0, the motion information for each sub-block can be derived using the juxtaposed image and the juxtaposed position with motion of 0.

[0209] When using multiple reference images, the juxtaposed images of different coding units in ATMVP may differ from each other. The fact that different coding units in the current image have different juxtaposed images for deriving ATMVP means that motion information fields from multiple reference images should be derived, which is not preferable due to increased storage bandwidth.

[0210] Therefore, this disclosure provides a simpler design for using the same juxtaposed image when deriving ATMVP. For example, a method for using the same juxtaposed image can be defined in the slice (or tile group) header, but this disclosure is not limited thereto. As an example, at the block level, when the reference image of a neighboring block A differs from the juxtaposed image, the motion vector of the neighboring block A can be scaled based on a time-motion vector scaling method. Additionally, the scaled motion vector of the neighboring block A can be used in the ATMVP.

[0211] Figure 17 This is a diagram illustrating a method for deriving advanced time motion vector prediction (ATMVP) candidates as an implementation of this disclosure.

[0212] Reference Figure 17In embodiments of this disclosure, since the TMVP using the motion vector of the lower right block of the current block or the time neighboring block (or colPB) at the center of the current block does not reflect in-image motion, the encoder / decoder can use the motion vector of colPB at the location indicated by the motion vector of the neighboring block as the MVP.

[0213] For example, encoders / decoders can be like Figure 17 The illustrated merge candidate configuration checks blocks in the same order, while finding the motion vector of the first available space neighboring block. Additionally, the positions indicated by the motion vectors in the reference image can be derived as col-PB (i.e., ATMVP candidates).

[0214] Additionally, motion vectors can be used as motion vectors for the corresponding block, with each sub-block as the unit. In this case, when there is no motion vector in a particular sub-block, the motion vector of the center block located at the center of the corresponding block can be used as the motion vector of the unavailable sub-block and stored as a representative motion vector.

[0215] Time motion vector data storage reduced

[0216] The embodiments of this disclosure propose a method to increase hardware cache hit rate when obtaining motion information stored in memory (e.g., DRAM) by referring only to motion information of a fixed reference image during the application of ATMVP.

[0217] The following implementation illustrates an example of deriving the processing of sub-block-based temporal merging underlying motion data. Hereinafter, spatial candidates or temporal candidates may be referred to as spatial merging candidates or temporal merging candidates.

[0218] Figure 18 An example is illustrated of a method for obtaining motion information from a predefined reference image based on an embodiment of the present disclosure in order to decode the current image.

[0219] Only use such Figure 18 The example in the text uses a predefined reference image for reference, such as... Figure 17 During the process of determining the spatial candidate position of the current block as illustrated in the example, the ATMVP is modified in a hardware-friendly manner based on whether the motion information of the corresponding spatial candidate references a predefined reference image.

[0220] Figure 19 An example configuration of a reference list as an embodiment of this disclosure is shown. (See also...) Figure 19 When the current image's picture order number (POC) is 4 in a random access structure with a group of pictures (GOP) of 16, the reference list of the current image will be described as an example.

[0221] When the current image (POC 4) is a B slice, the spatial reference candidates for the current image can be decoded by bidirectional prediction with reference to both the largest list 0 (L0) and list 1 (L1), or by unidirectional prediction with reference only to L0 or unidirectional prediction with reference only to L1.

[0222] Figure 20 An example of the configuration of reference images for each prediction mode and each prediction direction for neighboring blocks adjacent to the current block is illustrated as an implementation of this disclosure.

[0223] Reference Figure 20 Candidate 0 (cand0) is encoded / decoded via inter-frame prediction (bidirectional prediction), referencing POC(8) in the L0 direction and POC(0) in the L1 direction. Additionally, in Figure 20 In the process, candidate 1 (cand1) is encoded / decoded by intra-frame prediction, candidate 2 (cand2) is encoded / decoded by inter-frame prediction (one-way prediction), and POC (8) is referenced in the L1 direction. Candidate 3 (cand3) is encoded / decoded by intra-frame prediction, candidate 4 (cand4) is encoded / decoded by inter-frame prediction (one-way prediction), and POC (0) is referenced in the L0 direction.

[0224] In this case, when traversing the reference candidates, if the reference image in the corresponding direction of the reference candidate is a predefined reference image when traversing the reference candidates in the order of L0 and L1, then the embodiments of this disclosure propose a method for deriving ColPB based on the motion information of the reference candidates. In other words, when such... Figure 19 When decoding the proximity information of the current block, as illustrated in the example, if the POC of the predefined image is 8, then the POC-#8 image exists in both list 0 and list 1. In other words, in the process of determining the Collateral Predictive Block (ColPB) by examining the motion information of the neighboring spatial candidates in order to apply ATMVP to the current block, using the first candidate of the reference predefined image as motion information, a method is needed to efficiently determine the order of the list orientations or the search order.

[0225] In other words, when the reference image referenced by the spatial merge candidate is the same as the predefined reference image, the first merge candidate indicating the juxtaposition block specified by the motion information of the spatial merge candidate can be added to the spatial merge candidate list.

[0226] Subsequently, the merge index indicating a specific merge candidate in the merge candidate list is decoded, and the motion information of the specific merge candidate indicated by the merge index is used to generate the prediction block for the current block. Here, when the merge index indicates the first merge candidate, the prediction block can be generated by using the motion information of the sub-blocks of the first merge candidate.

[0227] Alternatively, spatial merge candidates can be selected from the merge candidate list based on the search order. Here, determining the search order can mean determining the search order of spatial merge candidates previously added to the merge candidate list.

[0228] Implementations of this disclosure propose a method for examining reference images in L0 and L1 order while traversing predefined reference candidates in order to generate a list of spatial candidates for inter-frame prediction of the current block.

[0229] Figure 21 This is an example illustrating a search order for generating a list of spatial candidates according to an embodiment of the present disclosure. Figure 21 Examples are shown for use with, for example Figure 20 The example in the text uses information about decoded neighboring blocks to generate the search order for the spatial candidate list.

[0230] The order of selection space candidates according to embodiments of the present disclosure will be described below.

[0231] like Figure 20 As illustrated, since candidate 1 and candidate 3 are decoded in intra-frame mode, candidate 1 and candidate 3 are excluded from this order, and when... Figure 21 When searching for candidate 1 and candidate 3 in the sequence illustrated, if the predefined POC of the image is 8, the co-placed prediction block (ColPb) is derived based on the motion vector (MV) of the list 0 of candidate 0 in sequence 0. This implementation can be carried out in the form of the code shown in Table 2.

[0232] [Table 2]

[0233]

[0234] The embodiments of this disclosure provide the following method: when traversing reference candidates in the order of reference directions L0 and L1, and a specific direction reference image of a specific reference candidate is a predefined reference image, ColPB is derived based on the motion information of the reference candidate referencing the predefined reference image.

[0235] When the current image (POC 4) is a B slice, the spatial reference candidates for the current image can be decoded by bidirectional prediction with reference to both the largest list 0 (L0) and list 1 (L1), or by unidirectional prediction with reference only to L0 or unidirectional prediction with reference only to L1.

[0236] In embodiments of this disclosure, the search order of spatial candidate blocks can be determined based on the tile type of the current image. For example, the sequential search condition may include whether the current image is a B tile.

[0237] Figure 22 Another example of a search order for generating a list of spatial candidates according to an embodiment of the present disclosure is illustrated. Figure 22 Examples are shown for use with, for example Figure 20 An example of how information about neighboring blocks is configured to generate the search order for a list of spatial candidates.

[0238] Reference Figure 22 Since the search is performed preferentially according to the reference directions (L0 and L1), the search is performed in the order of candidate 0 and candidate 1 with the predicted direction L0, and then in the order of candidate 0 and candidate 2 with the predicted direction L1.

[0239] Reference Figure 20 Since candidate 1 and candidate 3 are decoded in intra-frame mode, candidate 1 and candidate 3 are excluded from this order, and when... Figure 22 When searching for candidate 1 and candidate 3 in the sequence illustrated, if the predefined POC of the image is 8, the co-placed prediction block (ColPb) is derived based on the motion vector (MV) of the list 0 of candidate 0 in sequence 0. This implementation can be carried out in the form of the code shown in Table 3.

[0240] [Table 3]

[0241]

[0242] Figure 23 Another example of the configuration of a list of reference pictures as an embodiment of the present disclosure is shown.

[0243] In embodiments of this disclosure, when the current image is a B slice, the spatial reference candidates of the current image can be decoded by bidirectional prediction with reference to both the maximum list 0 (L0) and list 1 (L1), or by unidirectional prediction with reference only to L0 or unidirectional prediction with reference only to L1.

[0244] Implementations of this disclosure propose a method for determining the order of reference candidates by referring to a list of reference images configured for the current slice. For example... Figure 23As illustrated, a method can be used to check a specific orientation by first considering the case where the value of the POC configured in the reference list of the current image is equal to or less than the value of the current image in a low-latency scenario or in random access (i.e., only by considering the case of the image configuration list displayed before the current image).

[0245] refer to Figure 23 The L0 direction reference image (the reference list corresponding to the L0 direction) of the current image (POC(3)) can include POC(0) and POC(1), and the L1 direction reference image (the reference list corresponding to the L1 direction) of the current image (POC(3)) can include POC(0).

[0246] This implementation can be achieved using the codes shown in Tables 4 and 5.

[0247] [Table 4]

[0248]

[0249] In the code in Table 4, the function getCheckLDC() can be implemented in the form shown in Table 5, and this determines whether to make a low-latency condition based on the POC of the reference image list of the current image.

[0250] [Table 5]

[0251]

[0252] Figure 24 Another example is illustrated, which is an implementation of the present disclosure, of the configuration of reference images for each prediction mode and each prediction direction for neighboring blocks adjacent to the current block.

[0253] Reference Figure 24 Candidate 0 (cand0) is encoded / decoded via inter-frame prediction (bidirectional prediction), referencing POC(0) in both the L0 and L1 directions. Additionally, in Figure 20 In the process, candidate 1 (cand1) is encoded / decoded by intra-frame prediction, candidate 2 (cand2) is encoded / decoded by inter-frame prediction (one-way prediction), and POC (0) is referenced in the L1 direction. Candidate 3 (cand3) is encoded / decoded by intra-frame prediction, candidate 4 (cand4) is encoded / decoded by inter-frame prediction (one-way prediction), and POC (1) is referenced in the L0 direction.

[0254] Figure 25An example is given of a search order determined based on whether a reference image has a lower Picture Order Number (POC) than the current image, according to an embodiment of the present disclosure, for generating a spatial candidate list.

[0255] According to embodiments of this disclosure, although in Figure 23 Under the conditions illustrated in the example Figure 25 The sequential traversal shown in the example is as follows: Figure 24 The neighboring blocks are configured as an example, but when the reference image for a specific reference candidate in a specific direction is a predefined reference image, ColPB can be derived based on the motion information of the corresponding specific reference candidate.

[0256] Implementations of this disclosure propose a method for adaptively selecting an order from L0 to L1 or from L1 to L0 when traversing reference candidates by considering whether the reference images are derived by L0 or L1 according to a high-level syntax. For example, checking in which direction predefined list image information in the slice parameter set (SPS), picture parameter set (PPS), or slice header was generated allows for performing a search from the corresponding direction. For instance, the search order for selecting spatial candidates in a list of spatial candidates can be based on a specific list of reference images defined by a high-level syntax.

[0257] Figure 26 and Figure 27 Examples of search orders configured by taking into account priority directions defined in a higher-level syntax, according to embodiments of the present disclosure, are illustrated.

[0258] When Figure 20 When configuring neighboring blocks as illustrated in the example, if list 0 is a predefined list of images, then you can press... Figure 26 The search is performed in the order illustrated in the example. Since list 0 is a predefined list of images, the L0 direction of candidate 0 and the L0 direction of candidate 4, which are candidates with L0 direction reference images, are searched first. Then, the L1 direction of candidate 0 and the L1 direction of candidate 2, which are candidates with L1 direction reference images, are searched.

[0259] Conversely, when L1 is a predefined list of images, it can be pressed... Figure 27 The search is performed in the order illustrated in the example. Since List 1 is a predefined list of images, the L1 direction of Candidate 0 and Candidate 2, which are candidates with L1 direction reference images, is searched first. Then, the L0 direction of Candidate 0 and Candidate 4, which are candidates with L0 direction reference images, is searched.

[0260] In embodiments of this disclosure, all the above-mentioned embodiments are combined to determine the search order. In other words, the search order can be determined based on conditions including: the slice type of the current block, whether the POC of all reference images included in the reference image list defined by the high-level syntax is equal to or less than the POC of the current image. For example, the search order for selecting spatial candidates in a spatial candidate list can be based on whether the POC of all reference images included in the reference image list of the current image is equal to or less than the POC of the current image.

[0261] The search order for considering the conditions can be determined by the code shown in Table 6 below.

[0262] [Table 6]

[0263]

[0264] Alternatively, in other words, the search order can be determined based on at least one of the following: whether the POC of all reference images included in the current block's slice type, the reference image list defined by the high-level syntax, or the reference image list of the current image is equal to or less than the POC of the current image.

[0265] In implementations, temporal motion vectors can be derived from a reference image. Besides the encoder, temporal motion vectors can also be derived from all reference images that have been decoded by the decoder. However, the method used to derive all reference images may cause storage usage problems and increased storage bandwidth problems in the hardware that uses a temporal storage structure to store motion data of reference images for the current image tile, picture, and tile. As a method proposed in the embodiments of this disclosure, a method for deriving temporal motion vectors from predefined images is proposed. Figure 18 The example illustrates a method for deriving time motion vectors by referring to a reference image.

[0266] like Figure 18 As illustrated, although there are alternative reference images for deriving time motion vectors, the time motion data is derived from reference images predefined by a specific method.

[0267] Embodiments of this disclosure can provide a method for selecting a reference image to derive a time motion vector via a signaling syntax. The syntax can be signaled from the encoder to select a specific reference image from a list of images included in the reference image list. Table 7 illustrates the proposed syntax.

[0268] [Table 7]

[0269] descriptor if(temporaLmotion_verctor_prediction_enable) ref_idx_for_termporal_motion_vector_prediction u(1)

[0270] Specifically, the proposed syntax can be represented as shown in Table 8. Table 8 illustrates the syntax for deriving juxtaposed images in High Efficiency Video Coding (HEVC) to derive temporal motion vectors.

[0271] [Table 8]

[0272]

[0273] Figure 28 and Figure 29 An example of a reference image predefined in consideration of the image sequence number (POC) of the current image and the reference image is shown as an embodiment of the application of this disclosure.

[0274] In embodiments of this disclosure, a reference image for deriving the time motion vector can be defined by a decoder and an encoder according to the image sequence number (POC). The reference image can be defined as the image in a list that is closest to the current image. The image in the list that is closest to the current image has the smallest POC difference with it.

[0275] exist Figure 28 The example shown illustrates a scenario where an image with a minimum proof-of-concept (POC) is configured as a predefined reference image. (See reference...) Figure 28 The reference image (POC(4)) of the POC (POC(2), POC(3) and POC(4)) included in the L0 reference list that has the closest POC to the current image (POC(6)) is determined as the predefined reference image.

[0276] When reference images with the same POC distance exist in the list, select the reference image with the smallest Time ID (TID), such as... Figure 29 exemplified in .

[0277] Reference Figure 29 Among the reference images (POC(2), POC(3), and POC(4)) included in the L0 reference list, reference image (POC(4)) has the closest POC distance of 2 to the current image (POC(6)), and reference image (POC(8)) included in the L1 reference list also has the closest POC distance of 2 to the current image (POC(6)). Among the current image (reference block (POC(4)) of the L0 reference list with the closest POC distance of 2 to POC(6)) and reference block (POC(8)) of the L1 reference list), POC(8) with a smaller TID(0) can replace POC(4) with TID(3) and be determined as the predefined reference image.

[0278] Figure 30 and Figure 31Examples of methods for selecting space candidates for ATMVP as an implementation of this disclosure are illustrated.

[0279] In embodiments of this disclosure, a spatial candidate selection method for Adaptive Temporal Motion Vector Prediction (ATMVP) can be applied. ATMVP is one of the tools that uses temporal motion data from a reference image. A typical ATMVP derives temporal motion data from corresponding blocks. The position of the corresponding block is determined by the motion data of spatially neighboring candidates. This is because a typical ATMVP refers to the first available spatial candidate in the spatial candidate list order, in order to derive motion data from a reference spatial candidate that has both temporal and random motion data.

[0280] Therefore, reference images are defined using the selected candidate motion data to derive the temporal motion vector derivation method. In the worst case, the number of reference images is defined by the level capability of the compression standard used for image decoding.

[0281] In embodiments of this disclosure, such as Figure 30 and Figure 31 As illustrated, an improved ATMVP method using a reference image can be applied. In embodiments of this disclosure, the candidate order can be defined as an order from A to E, such as... Figure 30 exemplified in . like Figure 30 As illustrated, a spatial candidate has one or more reference images.

[0282] Reference Figure 30 Except for the unavailable block E, each of the spatial candidates A, B, C, and D adjacent to the current block references one or more reference images.

[0283] From the embodiments of this disclosure, spatial candidates that reference the same reference image as a predefined reference image can be selected as candidates for ATMVP, and can be as follows: Figure 31 The example demonstrates the derivation of time motion data.

[0284] Reference Figure 31 The spatial candidate B of the reference image corresponding to ref_idx(0) can be selected as a candidate for ATMVP, and the temporal motion data (MV) of the spatial candidate B can be derived. x b ,MV y b ).

[0285] In addition, for cases where there are no candidate reference images that are the same as the predefined reference images, the following three methods can be used.

[0286] 1) The zero motion vector is used to determine the corresponding block position, and the time motion vector is derived using sub-blocks as units.

[0287] 2) The use of ATMVP is configured to be disabled.

[0288] 3) Scale the motion vector of the first available space candidate to fit the predefined image.

[0289] Figure 32 An example is illustrated as an embodiment of the present disclosure of a method for deriving a temporal motion vector by scaling a spatial candidate motion vector.

[0290] like Figure 32 The example shown is represented as "MV". x a ,MV y a The motion vector of the first available spatial candidate A is scaled by a predetermined motion vector and determined as the "scaled MV". x a , scaled-up MV y a ".

[0291] Figure 33 and Figure 34 An example is given of a method for deriving time motion vectors based on backward mapping as an implementation of the present disclosure.

[0292] In embodiments of this disclosure, time motion vector derivation based on a backward mapping from a reference image can be applied. For example... Figure 33 The example shown illustrates a backward-mapped temporal motion data derivation method that can utilize motion data from all reference images.

[0293] One-sided candidate is a common back-mapping method. Even if there are several blocks without temporal motion data through the back-mapping process, the back-mapping process can be applied only to a predetermined image that includes the one-sided candidate method.

[0294] exist Figure 34 The example illustrates the one-sided candidate method described above (applying backward mapping by referring only to a predetermined reference image). See also... Figure 34 It is possible to apply the temporal motion data of a predetermined reference image from the reference images included in the L0 reference list to the current image through backward mapping.

[0295] Figure 35 This is a flowchart of generating a prediction block according to an embodiment of the present disclosure.

[0296] Reference Figure 35For ease of description, the decoder has been described in detail, but this disclosure is not limited thereto, and the method of generating inter-frame prediction blocks based on inter-frame prediction can be applied substantially equivalently to the encoder.

[0297] The decoder generates a list of merge candidates by using the spatial merge candidates and temporal merge candidates of the current block (S3501).

[0298] The decoder obtains the merge index indicating a specific merge candidate from the merge candidate list (S3502).

[0299] The decoder generates a prediction block for the current block by using motion information of specific merging candidates (S3503).

[0300] As an implementation method, the generation of the merging candidate list may include: when the reference image of the spatial merging candidate is the same as the predetermined image, adding the first merging candidate, which indicates the juxtaposition block specified by the motion information of the spatial merging candidate, to the merging candidate list.

[0301] As an implementation, the generation of the merge candidate list may further include determining the search order of spatial merge candidates previously added to the merge candidate list, and selecting spatial merge candidates from the merge candidate list based on the search order.

[0302] As an implementation method, the search order can be determined based on at least one of the following: the slice type of the current block, a specific list of reference images defined by the high-level syntax, and whether the picture sequence number (POC) of all reference images included in the current image's list of reference images is equal to or less than the POC of the current image.

[0303] As an implementation method, when the merge index indicates a first merge candidate, a prediction block can be generated by using the motion information of the sub-blocks of the first merge candidate.

[0304] As an implementation, the generation of the merge candidate list may include: when the reference image of the spatial merge candidate is different from the predetermined image, adding a second merge candidate, which indicates another juxtaposed block specified by scaling the motion information of the spatial merge candidate, to the merge candidate list.

[0305] Figure 36 This is a diagram illustrating an inter-frame prediction apparatus according to an embodiment of the present disclosure.

[0306] exist Figure 36 In this document, for ease of description, the inter-frame prediction unit is exemplified as a block, but it can be implemented as a component included in the encoder and / or decoder.

[0307] Reference Figure 36 Inter-frame prediction unit implements the above Figures 8 to 34The functions, processes, and / or methods proposed in the document. Specifically, the inter-frame prediction unit can be configured to include a merge candidate list generation unit 3601, a merge index acquisition unit 3602, and a prediction block generation unit 3603.

[0308] The merge candidate list generation unit 3601 generates a merge candidate list by using the spatial merge candidate and temporal merge candidate of the current block.

[0309] Merge index obtaining unit 3602 obtains a merge index indicating a specific merge candidate from the merge candidate list.

[0310] The prediction block generation unit 3603 generates a prediction block for the current block by using motion information of specific merging candidates.

[0311] As an implementation, the merge candidate list generation unit can be configured to add a first merge candidate, which indicates the juxtaposition block specified by the motion information of the spatial merge candidate, to the merge candidate list when the reference image of the spatial merge candidate is the same as the predetermined image.

[0312] As an implementation, the merge candidate list generation unit can be configured to determine the search order of spatial merge candidates previously added to the merge candidate list, and can select spatial merge candidates from the merge candidate list based on the search order.

[0313] As an implementation method, the search order can be determined based on at least one of the following: the slice type of the current block, a specific list of reference images defined by the high-level syntax, and whether the picture sequence number (POC) of all reference images included in the current image's list of reference images is equal to or less than the POC of the current image.

[0314] As an implementation method, when the merge index indicates a first merge candidate, a prediction block can be generated by using the motion information of the sub-blocks of the first merge candidate.

[0315] As an implementation, the merge candidate list generation unit can be configured to add a second merge candidate, which indicates another juxtaposed block specified by scaling the motion information of the spatial merge candidate, to the merge candidate list when the reference image of the spatial merge candidate is different from the predetermined image.

[0316] For ease of description, the embodiments of this disclosure described above have been described individually, but this disclosure is not limited thereto. That is, the above embodiments can be performed independently, and one or more of the various embodiments can be combined and performed.

[0317] Figure 37 An example of a video coding system using this disclosure is provided.

[0318] A video encoding system may include a source device and a receiving device. The source device may transmit encoded video / image information or data to the receiving device in the form of a file or stream via a digital storage medium or network.

[0319] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be configured as a separate device or an external component.

[0320] Video sources can acquire video / images through video / image capture, compositing, or generation processes. Video sources may include video / image capture devices and / or video / image generation devices. Video / image capture devices may include, for example, one or more cameras, video / image archives including previously captured video / images, etc. Video / image generation devices may include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images can be generated by computers, etc., and in this case, processing to generate relevant data can replace video / image capture processing.

[0321] Encoding devices can encode input video / images. They can perform a series of processes, including prediction, transformation, and quantization, to optimize compression and coding efficiency. The encoded data (encoded video / image information) can be output as a bitstream.

[0322] The transmitter can transmit encoded video / image information or data, output as a bitstream, to the receiver of the receiving device in the form of a file or stream via digital storage media or a network. Digital storage media can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter can include elements for generating media files according to a predetermined file format and may include elements for transmission via a broadcast / communication network. The receiver can extract the bitstream and send the extracted bitstream to a decoding device.

[0323] Decoding devices perform a series of processes corresponding to the operations of encoding devices, including inverse quantization, inverse transform, prediction, etc., to decode video / images.

[0324] The renderer can render the decoded video / image. The rendered video / image can then be displayed by a display unit.

[0325] Figure 38 This is a structural diagram of a content streaming system as an embodiment of the present disclosure.

[0326] Reference Figure 38 The content streaming system using this disclosure can mainly include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[0327] The encoding server compresses content input from multimedia input devices, including smartphones, cameras, and camcorders, into digital data to generate a bitstream, and then transmits this bitstream to a streaming server. As another example, when multimedia input devices, including smartphones, cameras, and camcorders, directly generate the bitstream, the encoding server can be omitted.

[0328] Bitstreams can be generated by applying the encoding method or bitstream generation method disclosed herein, and the streaming server can temporarily store bitstreams during the processing of sending or receiving bitstreams.

[0329] A streaming server sends multimedia data to a user device based on a user's request through a web server, and the web server acts as an intermediary to inform the user what services are available. When a user requests a desired service from the web server, the web server transmits the requested service to the streaming server, and the streaming server sends the multimedia data to the user. In this scenario, the content streaming system may include a separate control server, which in this case controls the commands / responses between the corresponding devices in the content streaming system.

[0330] A streaming server can receive content from media storage and / or encoding servers. For example, when a streaming server receives content from an encoding server, it can receive the content in real time. In this case, the streaming server can store the bitstream for a predetermined period of time to provide a smooth streaming service.

[0331] Examples of user devices may include cellular phones, smartphones, laptops, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, touchscreen PCs, tablet PCs, ultrabooks, and wearable devices such as smartwatches, smart glasses, and head-mounted displays (HMDs).

[0332] In a content streaming system, each server can operate as a distributed server, in which case the data received by each server can be processed in a distributed manner.

[0333] As described above, the embodiments described in this disclosure can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units illustrated in each figure can be implemented and executed on a computer, processor, microprocessor, controller, or chip.

[0334] Furthermore, the decoders and encoders disclosed herein can be included in multimedia broadcasting transmitting and receiving devices, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video chat devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, video-on-demand (VoD) service providers, (overhead) OTT video devices, internet streaming service providers, three-dimensional (3D) video devices, video telephony devices, transportation terminals (e.g., vehicle terminals, aircraft terminals, ship terminals, etc.), medical video devices, etc., and can be used to process video signals or data signals. For example, overhead (OTT) video devices can include game consoles, Blu-ray players, internet access TVs, home theater systems, smartphones, tablet PCs, digital video recorders (DVRs), etc.

[0335] Furthermore, the processing methods of this disclosure can be generated in the form of a program executed by a computer and can be stored in a computer-readable recording medium. Multimedia data having the data structure according to this disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices and distributed storage devices that store computer-readable data. Computer-readable recording media can include, for example, Blu-ray discs (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disks, and optical data storage devices. Additionally, computer-readable recording media include media implemented in the form of a carrier wave (e.g., transmission over the Internet). Furthermore, bitstreams generated by encoding methods can be stored in a computer-readable recording medium or transmitted via wired / wireless communication networks.

[0336] Furthermore, the embodiments of this disclosure can be implemented as a computer program product using program code, which can be executed on a computer using the embodiments of this disclosure. The program code can be stored on a computer-readable medium.

[0337] In the above embodiments, the components and features of this disclosure are combined in a predetermined manner. Unless otherwise expressly stated, each component or feature should be considered optional. Each component or feature may be implemented without association with other components or features. Additionally, embodiments of this disclosure can be configured by associating certain components and / or features. The order of operations described in embodiments of this disclosure may be changed. Some components or features of any embodiment may be included in another embodiment, or replaced by components and features corresponding to another embodiment. It is apparent that embodiments can be formed by combining claims not expressly referenced in the claims or by including them in new claims through post-submission modifications.

[0338] The embodiments of this disclosure can be implemented in hardware, firmware, software, or a combination thereof. In the case of hardware implementation, the exemplary embodiments described herein can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc., depending on the hardware implementation method.

[0339] When implemented by firmware or software, embodiments of this disclosure can be implemented as modules, processes, functions, etc., that perform the above-described functions or operations. Software code can be stored in memory and executed by a processor. The memory can be located internally or externally to the processor and can send data to / receive data from the processor using various known means.

[0340] It will be apparent to those skilled in the art that this disclosure may be practiced in other specific forms without departing from its essential characteristics. Therefore, the detailed description mentioned above should not be construed as limiting in any way and should be considered illustratively. The scope of this disclosure should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of this disclosure are included within its scope.

[0341] [Industrial Applicability]

[0342] For illustrative purposes, preferred embodiments of the present disclosure have been disclosed above. In the following text, various other embodiments may be modified, altered, substituted, or added by those skilled in the art within the technical spirit and scope of the present disclosure as disclosed in the appended claims.

Claims

1. An apparatus for decoding images based on inter-frame prediction modes, the apparatus comprising: A memory configured to store the image; as well as The processor, which is connected to the memory, The processor is configured as follows: Obtain the motion vector and reference image index of the left neighboring block of the current block; The juxtaposition block is determined based on the left neighbor block, wherein the juxtaposition block is determined based on the reference image represented by the reference image index; Generate a merge candidate list, wherein the merge candidate list includes the juxtaposition block as a time merge candidate; Obtain the merge index indicating the merge candidate in the merge candidate list; The predicted block for the current block is generated based on the motion information of the merged candidates; and The reconstructed block of the current block is generated based on the predicted block of the current block. Wherein, based on the fact that the L0 reference image of the left neighboring block is the same as the juxtaposed image, the juxtaposed block is set as the block within the juxtaposed image indicated by the L0 motion vector. Specifically, based on the fact that the L0 reference image is different from the juxtaposed image, the slice type of the current block is a B slice, and the L1 reference image of the left neighboring block is the same as the juxtaposed image, the juxtaposed block is set as the block within the juxtaposed image indicated by the L1 motion vector, and Wherein, the L0 reference image and the L1 reference image based on the left neighbor block are different from the juxtaposed image, and the juxtaposed block is set as the block indicated by the zero motion vector within the juxtaposed image.

2. The device according to claim 1, in, The juxtaposed block is determined solely based on the left neighboring block of the current block.

3. The device according to claim 1, in, The juxtaposed block is not determined based on the lower left neighboring block, upper right neighboring block, upper neighboring block, and upper left neighboring block of the current block.

4. The device according to any one of claims 1-3, in, The juxtaposed image is predefined by the slice header of the slice that includes the current block.

5. An apparatus for encoding images based on inter-frame prediction modes, the apparatus comprising: A memory configured to store the image; as well as The processor, which is connected to the memory, The processor is configured as follows: Obtain the motion vector and reference image index of the left neighboring block of the current block; The juxtaposition block is determined based on the left neighbor block, wherein the juxtaposition block is determined based on the reference image represented by the reference image index; Generate a merge candidate list, wherein the merge candidate list includes the juxtaposition block as a time merge candidate; Generate a merge index that indicates the merge candidates in the merge candidate list; The predicted block for the current block is generated based on the motion information of the merged candidates; and Generate the residual block of the current block based on the predicted block of the current block. Wherein, based on the fact that the L0 reference image of the left neighboring block is the same as the juxtaposed image, the juxtaposed block is set as the block within the juxtaposed image indicated by the L0 motion vector. Specifically, based on the fact that the L0 reference image is different from the juxtaposed image, the slice type of the current block is a B slice, and the L1 reference image of the left neighboring block is the same as the juxtaposed image, the juxtaposed block is set as the block within the juxtaposed image indicated by the L1 motion vector, and Wherein, the L0 reference image and the L1 reference image based on the left neighbor block are different from the juxtaposed image, and the juxtaposed block is set as the block indicated by the zero motion vector within the juxtaposed image.

6. The device according to claim 5, in, The juxtaposed block is determined solely based on the left neighboring block of the current block.

7. The device according to claim 5, in, The juxtaposed block is not determined based on the lower left neighboring block, upper right neighboring block, upper neighboring block, and upper left neighboring block of the current block.

8. The device according to any one of claims 5-7, in, The juxtaposed image is predefined by the slice header of the slice that includes the current block.

9. An apparatus for transmitting data for an image, the apparatus comprising: A processor configured to obtain a bitstream for the image; as well as A transmitter configured to transmit the data containing the bit stream. In order to obtain the bit stream, the processor is configured to: Obtain the motion vector and reference image index of the left neighboring block of the current block; The juxtaposition block is determined based on the left neighbor block, wherein the juxtaposition block is determined based on the reference image represented by the reference image index; Generate a merge candidate list, wherein the merge candidate list includes the juxtaposition block as a time merge candidate; Generate a merge index that indicates the merge candidates in the merge candidate list; The predicted block for the current block is generated based on the motion information of the merged candidates; and Generate the residual block of the current block based on the predicted block of the current block. Wherein, based on the fact that the L0 reference image of the left neighboring block is the same as the juxtaposed image, the juxtaposed block is set as the block within the juxtaposed image indicated by the L0 motion vector. Specifically, based on the fact that the L0 reference image is different from the juxtaposed image, the slice type of the current block is a B slice, and the L1 reference image of the left neighboring block is the same as the juxtaposed image, the juxtaposed block is set as the block within the juxtaposed image indicated by the L1 motion vector, and Wherein, the L0 reference image and the L1 reference image based on the left neighbor block are different from the juxtaposed image, and the juxtaposed block is set as the block indicated by the zero motion vector within the juxtaposed image.

10. The device according to claim 9, wherein: The juxtaposed block is determined solely based on the left neighboring block of the current block; or The juxtaposed block is not determined based on the lower left neighboring block, upper right neighboring block, upper neighboring block, and upper left neighboring block of the current block.

11. The device according to claim 9 or 10, in, The juxtaposed image is predefined by the slice header of the slice that includes the current block.

Images