Image encoding / decoding method, apparatus, and bitstream transmission method using a maximum size limit for chroma transformation blocks.

Abstract

To provide a method and apparatus for encoding / decoding an image.SOLUTION: The invention provides an image encoding / decoding method and apparatus. The image decoding method implemented with the image decoding apparatus may include determining a prediction mode of a current block, generating a prediction block of the current block based on inter prediction mode information if the prediction mode of the current block is an inter prediction mode, generating a residual block of the current block based on a transform block of the current block, and reconstructing the current block based on the prediction block and the residual block of the current block.SELECTED DRAWING: Figure 30

Description

Technical Field

[0001] The present disclosure relates to an image encoding / decoding method and apparatus, and more particularly, to a method for limiting the maximum size of a chroma conversion block and encoding / decoding an image, an apparatus, and a method for transmitting a bitstream generated by the image encoding method / apparatus of the present disclosure.

Background Art

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) images and UHD (Ultra High Definition) images, has been increasing in various fields. As image data becomes higher in resolution and quality, the amount of information or bits to be transmitted relatively increases compared to conventional image data. The increase in the amount of information or bits to be transmitted brings about an increase in transmission costs and storage costs.

[0003] Therefore, there is a need for a highly efficient image compression technology for effectively transmitting, storing, and reproducing information of high-resolution, high-quality images.

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present disclosure is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0005] Another object of the present disclosure is to provide an image encoding / decoding method and apparatus that improve encoding / decoding efficiency by limiting the maximum size of a chroma conversion block.

[0006] Another object of the present disclosure is to provide a method for transmitting a bitstream generated by an image encoding method or apparatus according to the present disclosure.

[0007] Furthermore, this disclosure aims to provide a recording medium that stores a bitstream generated by the image encoding method or apparatus according to this disclosure.

[0008] Furthermore, this disclosure aims to provide a recording medium that stores a bitstream received by the image decoding device provided herein, decoded, and used for image restoration.

[0009] The technical problems that this disclosure seeks to solve are not limited to those described above, and other technical problems not mentioned above will be clearly understood by a person with ordinary skill in the art to which this disclosure pertains from the following description. [Means for solving the problem]

[0010] An image decoding method performed by an image decoding apparatus according to one aspect of the present disclosure may include the steps of: determining the prediction mode of the current block; if the prediction mode of the current block is an inter-prediction mode, generating a prediction block for the current block based on inter-prediction mode information; generating a residual block for the current block based on a transformed block for the current block; and restoring the current block based on the prediction block and the residual block for the current block. In this case, the size of the transformed block may be determined based on the color components of the current block.

[0011] Furthermore, an image decoding apparatus according to one aspect of the present disclosure includes a memory and at least one processor, the at least one processor which determines the prediction mode of the current block, generates a prediction block for the current block based on the interprediction mode information if the prediction mode of the current block is an interprediction mode, generates a residual block for the current block based on the transformed block for the current block, and can restore the current block based on the prediction block and the residual block for the current block. In this case, the size of the transformed block can be determined based on the color components of the current block.

[0012] Furthermore, an image encoding method performed by an image encoding device according to one aspect of the present disclosure may include the steps of: dividing the image to determine a current block; generating an interprediction block of the current block; generating a residual block of the current block based on the interprediction block; and encoding interprediction mode information of the current block. In this case, the residual block is encoded based on the size of the transformation block of the current block, and the size of the transformation block can be determined based on the color components of the current block.

[0013] Furthermore, a transmission method according to another aspect of the present disclosure can transmit a bitstream generated by an image encoding device or image encoding method of the present disclosure.

[0014] Furthermore, a computer-readable recording medium according to another aspect of the present disclosure can store a bitstream generated by an image encoding method or image encoding apparatus of the present disclosure.

[0015] The features described above, which are a brief summary of this disclosure, are merely illustrative examples of the detailed description of this disclosure described below and do not limit the scope of this disclosure. [Effects of the Invention]

[0016] According to the present disclosure, it is possible to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0017] Further, according to the present disclosure, it is possible to provide an image encoding / decoding method and apparatus for improving encoding / decoding efficiency by restricting the maximum size of a chroma conversion block.

[0018] Further, according to the present disclosure, it is possible to provide a method for transmitting a bitstream generated by an image encoding method or apparatus according to the present disclosure.

[0019] Further, according to the present disclosure, it is possible to provide a recording medium storing a bitstream generated by an image encoding method or apparatus according to the present disclosure.

[0020] Further, according to the present disclosure, it is possible to provide a recording medium storing a bitstream received by an image decoding apparatus according to the present disclosure, decoded, and used for restoring an image.

[0021] The effects obtained in the present disclosure are not limited to the above-described effects, and other effects not described above will be clearly understood by those having ordinary knowledge in the technical field to which the present disclosure pertains from the following description.

Brief Description of the Drawings

[0022] [Figure 1] A diagram schematically showing a video coding system to which an embodiment according to the present disclosure is applicable. [Figure 2] A diagram schematically showing an image encoding apparatus to which an embodiment according to the present disclosure is applicable. [Figure 3] A diagram schematically showing an image decoding apparatus to which an embodiment according to the present disclosure is applicable. [Figure 4] A diagram showing a division structure of an image according to an embodiment. [Figure 5] A diagram showing an embodiment of a division type of a block according to a multi-type tree structure. [Figure 6] FIG. 1 is a diagram illustrating a signaling mechanism for block partition information in a quadtree with nested multi-type tree structure according to the present disclosure. [Figure 7] FIG. 2 shows an example in which a CTU is divided into multiple CUs. [Figure 8] FIG. 3 shows an example of a redundant partition pattern. [Figure 9] FIG. 4 is a flowchart showing an inter-prediction based video / image encoding method according to an embodiment. [Figure 10] FIG. 5 is a diagram exemplarily showing a configuration of an inter-prediction unit 180 according to an embodiment. [Figure 11] FIG. 6 is a flowchart showing an inter-prediction based video / image decoding method according to an embodiment. [Figure 12] FIG. 7 is a diagram exemplarily showing a configuration of an inter-prediction unit 260 according to an embodiment. [Figure 13] FIG. 8 is a diagram exemplifying peripheral blocks that can be used as spatial merge candidates according to an embodiment. [Figure 14] FIG. 9 is a diagram schematically showing a method for constructing a merge candidate list according to an embodiment. [Figure 15] FIG. 10 is a diagram schematically showing a method for constructing a motion vector predictor candidate list according to an embodiment. [Figure 16] FIG. 11 is a diagram showing a syntax structure for transmitting an MVD from an image encoding apparatus to an image decoding apparatus according to an embodiment. [Figure 17] FIG. 12 is a flowchart showing an IBC-based video / image encoding method according to an embodiment. [Figure 18] FIG. 13 is a diagram exemplarily showing a configuration of a prediction unit that performs an IBC-based video / image encoding method according to an embodiment. [Figure 19] FIG. 14 is a flowchart showing an IBC-based video / image decoding method according to an embodiment. [Figure 20]This figure illustrates the configuration of a prediction unit that performs an IBC-based video / image decoding method according to one embodiment. [Figure 21] This figure shows the syntax for chroma format signaling according to one embodiment. [Figure 22] This figure shows a chroma format classification table according to one example. [Figure 23] This diagram illustrates an example of the partitioning limitations for CUs (Units) for virtual pipeline processing. [Figure 24] This figure shows an example of the division of CU and TU according to one embodiment. [Figure 25] This figure shows an example of the division of CU and TU according to one embodiment. [Figure 26] This figure shows an example of the division of CU and TU according to one embodiment. [Figure 27] This flowchart shows inter-prediction and intra-prediction with the maximum conversion size applied according to one embodiment. [Figure 28] This flowchart shows inter-prediction and intra-prediction with the maximum conversion size applied according to one embodiment. [Figure 29] This is a flowchart illustrating how an encoding device according to one embodiment encodes an image. [Figure 30] This is a flowchart showing how a decoding device according to one embodiment decodes an image. [Figure 31] This figure illustrates a content streaming system to which the embodiments of this disclosure can be applied. [Modes for carrying out the invention]

[0023] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings, so that they can be easily implemented by a person with ordinary skill in the art to which the present disclosure pertains. However, the present disclosure can be implemented in a variety of different forms and is not limited to the embodiments described herein.

[0024] In describing embodiments of this disclosure, if it is determined that a specific description of a known configuration or function would obscure the gist of this disclosure, such detailed description will be omitted. In the drawings, parts unrelated to the description of this disclosure will be omitted, and similar parts will be denoted by the same reference numerals.

[0025] In this disclosure, when one component is described as being “connected,” “joined,” or “linked” to another component, this can include not only direct connections but also indirect connections where another component exists between them. Furthermore, when one component is described as “containing” or “having” another component, this means, unless otherwise stated to the contrary, that it may include another component rather than excluding it.

[0026] In this disclosure, terms such as "first," "second," etc., are used solely for the purpose of distinguishing one component from another, and do not limit the order or importance of the components unless otherwise specified. Therefore, within the scope of this disclosure, the first component of one embodiment may be called the second component in another embodiment, and similarly, the second component of one embodiment may be called the first component in another embodiment.

[0027] In this disclosure, components that are distinguished from each other are used to clearly describe their respective characteristics and do not necessarily mean that the components are separate. In other words, multiple components may be integrated to constitute a single hardware or software unit, or a single component may be distributed to constitute multiple hardware or software units. Therefore, such integrated or distributed embodiments are also included in the scope of this disclosure, without needing to be specifically mentioned.

[0028] In this disclosure, the components described in various embodiments are not necessarily essential components, and some may be optional components. Therefore, embodiments consisting of a subset of the components described in one embodiment are also included in the scope of this disclosure. Furthermore, embodiments that include additional components in addition to the components described in various embodiments are also included in the scope of this disclosure.

[0029] This disclosure relates to the encoding and decoding of images, and the terms used in this disclosure may have their ordinary meanings in the art to which this disclosure pertains, unless they are newly defined in this disclosure.

[0030] In this disclosure, "picture" generally means a unit representing any one image within a specific time period, and "slice / tile" is an encoding unit that constitutes part of a picture, and a single picture can consist of one or more slices / tiles. Furthermore, a slice / tile may contain one or more CTUs (coding tree units).

[0031] In this disclosure, “pixel” or “pel” may mean the smallest unit that constitutes a picture (or image). The term “sample” may also be used as a counterpart to pixel. A sample may generally represent a pixel or a pixel value, or it may represent only the pixel / pixel value of the luma component, or only the pixel / pixel value of the chroma component.

[0032] In this disclosure, “unit” can refer to a basic unit of image processing. A unit may include at least one of a specific region of a picture and information associated with that region. A unit may be used interchangeably with terms such as “sample array,” “block,” or “area,” as it may be used. Generally, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients consisting of M columns and N rows.

[0033] In this disclosure, “current block” can mean any one of the following: “current coding block,” “current coding unit,” “block to encode,” “block to decode,” or “block to process.” If prediction is performed, “current block” can mean “current prediction block” or “block to predict.” If transformation (inverse transformation) / quantization (inverse quantization) is performed, “current block” can mean “current transformation block” or “block to transform.” If filtering is performed, “current block” can mean “block to filter.”

[0034] Furthermore, in this disclosure, “current block” may mean “chroma block of the current block” unless there is an explicit mention of chroma block. “Chroma block of the current block” may be expressed explicitly as “chroma block” or “current chroma block,” including an explicit mention of chroma block.

[0035] In this disclosure, " / " and "," may be interpreted as "and / or." For example, "A / B" and "A, B" may be interpreted as "A and / or B." Also, "A / B / C" and "A, B, C" may mean "at least one of A, B and / or C."

[0036] In this disclosure, “or” may be interpreted as “and / or.” For example, “A or B” may mean 1) “A” only, 2) “B” only, or 3) “A and B.” Alternatively, in this disclosure, “or” may mean “additionally or alternatively.”

[0037] Overview of the video coding system

[0038] Figure 1 shows the video coding system according to this disclosure.

[0039] A video coding system according to one embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 can transmit encoded video and / or image information or data to the decoding device 20 via a digital storage medium or network in file or streaming format.

[0040] An encoding device 10 according to one embodiment may include a video source generation unit 11, an encoding unit 12, and a transmission unit 13. A decoding device 20 according to one embodiment may include a receiving unit 21, a decoding unit 22, and a rendering unit 23. The encoding unit 12 may be called a video / image encoding unit, and the decoding unit 22 may be called a video / image decoding unit. The transmission unit 13 may be included in the encoding unit 12. The receiving unit 21 may be included in the decoding unit 22. The rendering unit 23 may also include a display unit, which may be configured as a separate device or external component.

[0041] The video source generation unit 11 can acquire video / images through processes such as video / image capture, synthesis, or generation. The video source generation unit 11 may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, or a video / image archive containing previously captured video / images. The video / image generation device may include, for example, a computer, tablet, and smartphone, and may generate video / images (electronically). For example, virtual video / images may be generated via a computer, in which case the video / image capture process may be replaced by a process in which the relevant data is generated.

[0042] The encoding unit 12 can encode the input video / image. The encoding unit 12 can perform a series of steps such as prediction, transformation, and quantization for compression and encoding efficiency. The encoding unit 12 can output the encoded data (encoded video / image information) in bitstream format.

[0043] The transmission unit 13 can transmit encoded video / image information or data, output in bitstream format, to the receiving unit 21 of the decoding device 20 via a digital storage medium or network in file or streaming format. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray®, HDD, and SSD. The transmission unit 13 may include elements for generating media files via a predetermined file format and elements for transmission via a broadcast / communication network. The receiving unit 21 can extract / receive the bitstream from the storage medium or network and transmit it to the decoding unit 22.

[0044] The decoding unit 22 can decode the video / image by performing a series of steps such as inverse quantization, inverse transform, and prediction, corresponding to the operation of the encoding unit 12.

[0045] The rendering unit 23 can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0046] Overview of Image Encoding Devices

[0047] Figure 2 is a schematic diagram showing an image encoding device to which the embodiments of this disclosure can be applied.

[0048] As shown in Figure 2, the image coding device 100 may include an image splitting unit 110, a subtraction unit 115, a transformation unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse transformation unit 150, an addition unit 155, a filtering unit 160, a memory 170, an inter-prediction unit 180, an intra-prediction unit 185, and an entropy coding unit 190. The inter-prediction unit 180 and the intra-prediction unit 185 can together be called the "prediction unit". The transformation unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse transformation unit 150 may be included in a residual processing unit. The residual processing unit may further include a subtraction unit 115.

[0049] All or at least some of the multiple components constituting the image encoding device 100 can be implemented by a single hardware component (e.g., an encoder or processor) depending on the embodiment. Furthermore, the memory 170 may include a DPB (decoded picture buffer) and can be implemented by a digital storage medium.

[0050] The image splitting unit 110 can split an input image (or picture, frame) input to the image encoding device 100 into one or more processing units. For example, the processing units may be called coding units (CUs). Coding units can be obtained by recursively splitting a coding tree unit (CTU) or the largest coding unit (LCU) using a QT / BT / TT (Quad-tree / binary-tree / ternary-tree) structure. For example, a single coding unit can be split into multiple coding units of deeper depth based on a quad-tree structure, a binary-tree structure and / or a ternary-tree structure. For the splitting of coding units, a quad-tree structure may be applied first, followed by a binary-tree structure and / or a ternary-tree structure. Based on the final coding unit that cannot be further split, the coding procedure according to this disclosure can be performed. The largest coding unit can be used as the final coding unit, or a lower-depth coding unit obtained by dividing the largest coding unit can be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and / or restoration, as described later. As another example, the processing units of the coding procedure may be prediction units (PU) or transformation units (TU). The prediction unit and the transformation unit may be divided or partitioned from the final coding unit, respectively. The prediction unit may be a unit of sample prediction, and the transformation unit may be a unit that derives transformation coefficients and / or a unit that derives a residual signal from transformation coefficients.

[0051] The prediction unit (inter-prediction unit 180 or intra-prediction unit 185) can make predictions for the block to be processed (current block) and generate a predicted block that includes prediction samples for the current block. The prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block or on a CU basis. The prediction unit can generate various information regarding the prediction of the current block and transmit it to the entropy coding unit 190. The prediction information can be encoded by the entropy coding unit 190 and output in bitstream format.

[0052] The intra-prediction unit 185 can predict the current block by referring to a sample in the current picture. The referenced sample may be located in the vicinity (neighbor) or at a distance from the current block, according to the intra-prediction mode and / or intra-prediction technique. The intra-prediction mode may include multiple non-directional modes and multiple directional modes. The non-directional modes may include, for example, a DC mode and a Planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is merely an example, and more or fewer directional prediction modes may be used depending on the settings. The intra-prediction unit 185 may also determine the prediction mode to be applied to the current block using the prediction modes applied to the surrounding blocks.

[0053] The interprediction unit 180 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between the surrounding blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, the surrounding blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture containing the reference block and the reference picture containing the temporal neighboring block may be the same or different from each other. The temporal neighboring block may be called a collocated reference block, collocated CU (colCU), etc. The reference picture containing the temporal neighboring block may be called a collocated picture (colPic). For example, the interpretation unit 180 can construct a motion information candidate list based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Interpretation can be performed based on various prediction modes; for example, in skip mode and merge mode, the interpretation unit 180 can use the motion information of surrounding blocks as the motion information of the current block. In skip mode, unlike merge mode, the residual signal may not be transmitted.In motion vector prediction (MVP) mode, the motion vector of the surrounding block is used as the motion vector predictor, and the motion vector of the current block can be signaled by encoding the motion vector difference and an indicator for the motion vector predictor. The motion vector difference can represent the difference between the motion vector of the current block and the motion vector predictor.

[0054] The prediction unit can generate a prediction signal based on various prediction methods and / or techniques described later. For example, the prediction unit can apply intra-prediction or inter-prediction to predict the current block, and can also apply intra-prediction and inter-prediction simultaneously. A prediction method that applies intra-prediction and inter-prediction simultaneously to predict the current block can be called CIIP (combined inter and intra prediction). The prediction unit can also perform intra-block copy (IBC) to predict the current block. Intra-block copy can be used for content image / video coding such as in games, for example, in SCC (screen content coding). IBC is a method of predicting the current block using a reference block that has already been restored in the current picture at a predetermined distance from the current block. When IBC is applied, the position of the reference block in the current picture can be encoded as a vector (block vector) corresponding to the predetermined distance. IBC basically performs prediction within the current picture, but can be performed similarly to inter-prediction in that it derives the reference block within the current picture. In other words, IBC can use at least one of the interpretation techniques described in this disclosure.

[0055] The predicted signal generated by the prediction unit can be used to generate a reconstructed signal or a residual signal. The subtraction unit 115 can generate a residual signal (residual block, residual sample array) by subtracting the predicted signal output from the prediction unit (predicted block, predicted sample array) from the input image signal (original block, original sample array). The generated residual signal can be transmitted to the conversion unit 120.

[0056] The transformation unit 120 can generate transformation coefficients by applying transformation techniques to the residual signal. For example, the transformation techniques may include at least one of the following: DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT refers to a transformation obtained from a graph when the relationship information between pixels is represented by this graph. CNT refers to a transformation obtained by generating a prediction signal using all previously reconstructed pixels. The transformation process can be applied to pixel blocks of the same size and are square, or to non-square, variable-sized blocks.

[0057] The quantization unit 130 can quantize the conversion coefficients and transmit them to the entropy coding unit 190. The entropy coding unit 190 can encode the quantized signal (information about the quantized conversion coefficients) and output it in bitstream format. The information about the quantized conversion coefficients can be called residual information. The quantization unit 130 can rearrange the block-form quantized conversion coefficients into a one-dimensional vector format based on the coefficient scan order, and can also generate information about the quantized conversion coefficients based on the one-dimensional vector format of the quantized conversion coefficients.

[0058] The entropy coding unit 190 can perform various coding methods, such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding). The entropy encoding unit 190 can encode, together or separately, information necessary for video / image restoration (e.g., syntax elements (values ​​of syntax elements), in addition to the quantized conversion coefficients). The encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream format in units of network abstraction layer (NAL) units. The video / image information may further include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). The video / image information may also further include general constraint information. The signaling information, transmitted information, and / or syntax elements referred to in this disclosure can be encoded via the encoding procedure described above and included in the bitstream.

[0059] The bitstream can be transmitted over a network or stored on a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmission unit (not shown) for transmitting the signal output from the entropy encoding unit 190 and / or a storage unit (not shown) for storing it may be provided as an internal / external element of the image encoding device 100, or the transmission unit may be provided as a component of the entropy encoding unit 190.

[0060] The quantized conversion coefficients output from the quantization unit 130 can be used to generate a residual signal. For example, by applying inverse quantization and inverse transformation to the quantized conversion coefficients via the inverse quantization unit 140 and the inverse transformation unit 150, a residual signal (residual block or residual sample) can be reconstructed.

[0061] The adder 155 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter-prediction unit 180 or the intra-prediction unit 185. If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The adder 155 may be called the reconstruction unit or the reconstructed block generation unit. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, or, as described later, for inter-prediction of the next picture after filtering.

[0062] The filtering unit 160 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 160 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 170, specifically in the DPB of the memory 170. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter. The filtering unit 160 can generate various filtering-related information, as will be described later in the explanation of each filtering method, and transmit it to the entropy coding unit 190. The filtering-related information can be encoded by the entropy coding unit 190 and output in bitstream format.

[0063] The corrected restored picture transmitted to memory 170 can be used as a reference picture in the interpretation unit 180. When interpretation is applied via this, the image encoding device 100 can avoid prediction mismatches between the image encoding device 100 and the image decoding device, and can also improve encoding efficiency.

[0064] The DPB in memory 170 can store the modified restored picture for use as a reference picture in the inter-prediction unit 180. Memory 170 can store motion information of blocks from which motion information in the current picture has been derived (or encoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 180 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 170 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 185.

[0065] Overview of the image decoding device

[0066] Figure 3 is a schematic diagram showing an image decoding apparatus to which the embodiments of this disclosure can be applied.

[0067] As shown in Figure 3, the image decoding device 200 can be configured to include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an additive unit 235, a filtering unit 240, a memory 250, an inter-prediction unit 260, and an intra-prediction unit 265. The inter-prediction unit 260 and the intra-prediction unit 265 can together be called the "prediction unit". The inverse quantization unit 220 and the inverse transform unit 230 can be included in the residual processing unit.

[0068] All or at least some of the multiple components constituting the image decoding device 200 can be implemented by a single hardware component (e.g., a decoder or processor) according to the embodiment. Furthermore, the memory 170 may include a DPB and can be implemented by a digital storage medium.

[0069] An image decoding device 200, upon receiving a bitstream containing video / image information, can restore the image by executing a process corresponding to the process performed in the image encoding device 100 in Figure 1. For example, the image decoding device 200 can perform decoding using the processing unit applied in the image encoding device. Therefore, the decoding processing unit can be, for example, a coding unit. The coding unit can be obtained by dividing a coding tree unit or a maximum coding unit. The restored image signal decoded and output via the image decoding device 200 can then be reproduced via a playback device (not shown).

[0070] The image decoding device 200 can receive the signal output from the image encoding device 2 in bitstream format. The received signal can be decoded via the entropy decoding unit 210. For example, the entropy decoding unit 210 can parse the bitstream to derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information about various parameter sets, such as adaptive parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. The image decoding device may further use the parameter set information and / or the general constraint information to decode the image. The signaling information, received information, and / or syntax elements referred to in this disclosure can be obtained from the bitstream by decoding via the decoding procedure. For example, the entropy decoding unit 210 can decode information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements necessary for image reconstruction and the quantized values ​​of conversion coefficients related to the residual. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element from the bitstream, determines a context model using the syntax element information to be decoded, the decoding information of the surrounding blocks and the blocks to be decoded, or the symbol / bin information decoded in a previous step, predicts the probability of bin occurrence based on the determined context model, and performs arithmetic decoding of the bins to generate symbols corresponding to the values ​​of each syntax element. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the context model of the next symbol / bin.Of the information decoded by the entropy decoding unit 210, information related to prediction is provided to the prediction unit (inter-prediction unit 260 and intra-prediction unit 265), and the residual values ​​that have undergone entropy decoding in the entropy decoding unit 210, i.e., quantized conversion coefficients and related parameter information, can be input to the inverse quantization unit 220. In addition, of the information decoded by the entropy decoding unit 210, information related to filtering can be provided to the filtering unit 240. On the other hand, a receiving unit (not shown) that receives signals output from the image coding device may be further provided as an internal / external element of the image decoding device 200, or the receiving unit may be provided as a component of the entropy decoding unit 210.

[0071] On the other hand, the image decoding device according to this disclosure may be called a video / image / picture decoding device. The image decoding device may also include an information decoder (video / image / picture information decoder) and / or a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoding unit 210, and the sample decoder may include at least one of an inverse quantization unit 220, an inverse transform unit 230, an adder unit 235, a filtering unit 240, a memory 250, an inter-prediction unit 260, and an intra-prediction unit 265.

[0072] The inverse quantization unit 220 can inverse quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 220 can rearrange the quantized transformation coefficients in a two-dimensional block format. In this case, the rearrangement can be performed based on the coefficient scan order performed by the image encoding device. The inverse quantization unit 220 can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) to obtain the transformation coefficients.

[0073] The inverse conversion unit 230 can inversely convert the conversion coefficients to obtain residual signals (residual blocks, residual sample arrays).

[0074] The prediction unit can make predictions for the current block and generate a predicted block containing prediction samples for the current block. Based on the prediction information output from the entropy decoding unit 210, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block and can determine a specific intra / inter-prediction mode (prediction technique).

[0075] As described in the explanation of the prediction unit of the image coding device 100, the prediction unit can generate prediction signals based on various prediction methods (techniques) described later.

[0076] The intra-prediction unit 265 can predict the current block by referring to the samples in the current picture. The description of the intra-prediction unit 185 can also be applied to the intra-prediction unit 265.

[0077] The interprediction unit 260 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on a reference picture. In this case, to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in block, sub-block, or sample units based on the correlation of motion information between surrounding blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In interprediction, surrounding blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the interprediction unit 260 can construct a motion information candidate list based on surrounding blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction can be performed based on various prediction modes (techniques), and the prediction information may include information indicating the mode (technique) of interprediction for the current block.

[0078] The adder 235 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the prediction unit (including the inter-prediction unit 260 and / or intra-prediction unit 265). If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The description of the adder 155 can also be applied to the adder 235. The adder 235 can be called the reconstruction unit or reconstructed block generation unit. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, or, as described later, for inter-prediction of the next picture after filtering.

[0079] The filtering unit 240 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 240 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 250, specifically in the DPB of the memory 250. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.

[0080] The restored picture stored (modified) in the DPB of memory 250 can be used as a reference picture in the inter-prediction unit 260. Memory 250 can store motion information of blocks from which motion information in the current picture has been derived (or decoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 260 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 250 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 265.

[0081] In this specification, the embodiments described for the filtering unit 160, inter-prediction unit 180, and intra-prediction unit 185 of the image coding device 100 can be applied similarly or in a corresponding manner to the filtering unit 240, inter-prediction unit 260, and intra-prediction unit 265 of the image decoding device 200, respectively.

[0082] Overview of image segmentation

[0083] The video / image coding method according to this disclosure can be performed based on the following image segmentation structure. Specifically, procedures such as prediction, residual processing (inverse transformation, inverse quantization, etc.), syntax element coding, and filtering, described later, can be performed based on CTU, CU (and / or TU, PU) derived from the image segmentation structure. The image can be segmented into blocks, and the block segmentation procedure can be performed in the image segmentation unit 110 of the encoding device described above. Segmentation-related information can be encoded in the entropy encoding unit 190 and transmitted to the decoding device in bitstream format. The entropy decoding unit 210 of the decoding device can derive the block segmentation structure of the current picture based on the segmentation-related information obtained from the bitstream, and perform a series of procedures for image decoding (e.g., prediction, residual processing, block / picture reconstruction, in-loop filtering, etc.) based on this.

[0084] A picture can be divided into a sequence of coding tree units (CTUs). Figure 4 shows an example of a picture being divided into CTUs. A CTU can correspond to a coding tree block (CTB). Alternatively, a CTU can contain two coding tree blocks: one for the luma sample and one for the corresponding chroma sample. For example, for a picture containing three sample arrays, the CTU can contain an N×N block for the luma sample and two corresponding blocks for the chroma sample.

[0085] Overview of CTU division

[0086] As mentioned above, coding units can be obtained by recursively partitioning a coding tree unit (CTU) or a maximum coding unit (LCU) using a QT / BT / TT (Quad-tree / binary-tree / ternary-tree) structure. For example, a CTU can first be partitioned into a quadtree structure. Then, the leaf nodes of the quadtree structure can be further partitioned into a multi-type tree structure.

[0087] A quadtree partition means dividing the current CU (or CTU) into four equal parts. Through a quadtree partition, the current CU can be divided into four CUs of the same width and height. If the current CU is not further divided into a quadtree structure, it corresponds to a leaf node in the quadtree structure. A CU that corresponds to a leaf node in a quadtree structure is not further divided and can be used as the final coding unit as described above. Alternatively, a CU that corresponds to a leaf node in a quadtree structure can be further divided by a multi-type tree structure.

[0088] Figure 5 shows the types of block partitioning using a multi-type tree structure. Partitioning using a multi-type tree structure can include two partitions using a binary tree structure and two partitions using a ternary tree structure.

[0089] The two types of partitioning using a binary tree structure include vertical binary splitting (SPLIT_BT_VER) and horizontal binary splitting (SPLIT_BT_HOR). Vertical binary splitting (SPLIT_BT_VER) means splitting the current CU vertically into two equal parts. As shown in Figure 4, vertical binary splitting can generate two CUs that have the same height as the current CU and half the width of the current CU. Horizontal binary splitting (SPLIT_BT_HOR) means splitting the current CU horizontally into two equal parts. As shown in Figure 5, horizontal binary splitting can generate two CUs that have half the height of the current CU and the same width as the current CU.

[0090] Two types of partitioning using a ternary structure are vertical ternary splitting (SPLIT_TT_VER) and horizontal ternary splitting (SPLIT_TT_HOR). Vertical ternary splitting (SPLIT_TT_VER) divides the current CU vertically in a 1:2:1 ratio. As shown in Figure 5, vertical ternary splitting can produce two CUs with the same height as the current CU and a width of 1 / 4 of the current CU's width, and one CU with the same height as the current CU and a width of half the current CU's width. Horizontal ternary splitting (SPLIT_TT_HOR) divides the current CU horizontally in a 1:2:1 ratio. As shown in Figure 4, horizontal ternary splitting can produce two CUs with the same height as the current CU and a width of 1 / 4 of the current CU's width, and one CU with the same height as the current CU and a width of half the current CU's width.

[0091] Figure 6 illustrates the signaling mechanism for block partitioning information in a quadtree with nested multi-type tree structure according to this disclosure.

[0092] Here, the CTU is treated as the root node of the quadtree, and the CTU is the first node to be split into a quadtree structure. Information (e.g., qt_split_flag) indicating whether or not to split the quadtree can be signaled to the current CU (CTU or quadtree node (QT_node)). For example, if qt_split_flag is the first value (e.g., "1"), the current CU can be split into a quadtree. If qt_split_flag is the second value (e.g., "0"), the current CU will not be split into a quadtree and will become a leaf node (QT_leaf_node) of the quadtree. Each leaf node of the quadtree can subsequently be further split into a multitype tree structure. In other words, a leaf node of a quadtree can become a node (MTT_node) of a multitype tree. In a multi-type tree structure, a first flag (e.g., mtt_split_cu_flag) can be signaled to indicate whether the current node will be further split. If the node is to be further split (e.g., the first flag is 1), a second flag (e.g., mtt_split_cu_verticla_flag) can be signaled to indicate the splitting direction. For example, if the second flag is 1, the splitting direction is vertical, and if the second flag is 0, the splitting direction is horizontal. Subsequently, a third flag (e.g., mtt_split_cu_binary_flag) can be signaled to indicate whether the splitting type is binary or ternary. For example, if the third flag is 1, the splitting type is binary, and if the third flag is 0, the splitting type is ternary. Nodes in a multitype tree obtained by binary partitioning or ternary partitioning can be further partitioned into a multitype tree structure. However, nodes in a multitype tree cannot be partitioned into a quadtree structure.If the first flag is 0, the corresponding node in the multitype tree is not further subdivided and becomes a leaf node (MTT_leaf_node) of the multitype tree. A CU corresponding to a leaf node in the multitype tree can be used as the final coding unit as described above.

[0093] Based on the aforementioned mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree splitting mode (MttSplitMode) of the CU can be derived as shown in Table 1. In the following description, the multi-tree splitting mode may be abbreviated as the multi-tree splitting type or splitting type.

[0094] [Table 1]

[0095] Figure 7 shows an example where a CTU is divided into multiple CUs by applying a multitype tree after a quadtree. In Figure 7, the bold block edge 710 represents the quadtree division, and the remaining edge 720 represents the multitype tree division. A CU can correspond to a coding block CB. In one embodiment, a CU may include two coding blocks: a coding block for a luma sample and a coding block for a chroma sample corresponding to the luma sample. The chroma component (sample) CB or TB size can be derived based on the luma component (sample) CB or TB size according to the component ratio of the picture / image color format (chroma format, e.g., 4:4:4, 4:2:2, 4:2:0, etc.). If the color format is 4:4:4, the chroma component CB / TB size can be set to be the same as the luma component CB / TB size. If the color format is 4:2:2, the width of the chroma component CB / TB can be set to half the width of the luma component CB / TB, and the height of the chroma component CB / TB can be set to the height of the luma component CB / TB. If the color format is 4:2:0, the width of the chroma component CB / TB can be set to half the width of the luma component CB / TB, and the height of the chroma component CB / TB can be set to half the height of the luma component CB / TB.

[0096] In one embodiment, when the size of the CTU is 128 based on the luma sample unit, the size of the CU can range from 128 × 128, which is the same size as the CTU, to 4 × 4. In one embodiment, when the color format is 4:2:0 (or chroma format), the chroma CB size can range from 64 × 64 to 2 × 2.

[0097] On the other hand, in one embodiment, the CU size and TU size can be the same. Alternatively, multiple TUs can exist within the CU region. The TU size generally refers to the Luma component (sample) TB (Transform Block) size.

[0098] The TU size can be derived based on a preset value, the maximum allowable TB size (maxTbSize). For example, if the CU size is larger than the maxTbSize, multiple TUs (TBs) with the maxTbSize can be derived from the CU, and conversion / inverse conversion can be performed in units of the TUs (TBs). For example, the maximum allowable lumen TB size may be 64×64, and the maximum allowable chromen TB size may be 32×32. If the width or height of a CB divided by the tree structure is larger than the maximum conversion width or height, the CB can be automatically (or implicitly) divided until the horizontal and vertical TB size limits are satisfied.

[0099] Furthermore, for example, when intra-prediction is applied, the intra-prediction mode / type is derived on a CU (or CB) basis, and the peripheral reference sample derivation and prediction sample generation procedures can be performed on a TU (or TB) basis. In this case, one or more TUs (or TBs) can exist within a single CU (or CB) region, and in this case, the multiple TUs (or TBs) can share the same intra-prediction mode / type.

[0100] On the other hand, for a quadtree coding tree scheme with multitype trees, the following parameters can be signaled from the encoder to the decoder as SPS syntax elements. For example, at least one of the following can be signaled: CTUsize, which indicates the size of the root node of the quadtree; MinQTSize, which indicates the minimum allowed size of the leaf nodes of the quadtree; MaxBTSize, which indicates the maximum allowed size of the root node of the binary tree; MaxTTSize, which indicates the maximum allowed size of the root node of the ternary tree; MaxMttDepth, which indicates the maximum allowed hierarchy depth of the multitype trees that are split from the leaf nodes of the quadtree; MinBtSize, which indicates the minimum allowed leaf node size of the binary tree; and MinTtSize, which indicates the minimum allowed leaf node size of the ternary tree.

[0101] In one embodiment using the 4:2:0 chroma format, the CTU size can be set to a 128x128 chroma block and two corresponding 64x64 chroma blocks. In this case, MinQTSize can be set to 16x16, MaxBtSize to 128x128, MaxTtSzie to 64x64, MinBtSize and MinTtSize to 4x4, and MaxMttDepth to 4. A quadtree partition can be applied to the CTU to generate leaf nodes of the quadtree. Leaf nodes of a quadtree can be called leaf QT nodes. Leaf nodes of a quadtree can range in size from 16x16 (e.g., the MinQTSize) to 128x128 (e.g., the CTU size). If a leaf QT node is 128x128, it may not be further partitioned into a binary / ternary tree. This is because even if partitioned in this case, it would exceed MaxBtsize and MaxTtszie (e.g., 64x64). Otherwise, a leaf QT node can be further partitioned into a multitype tree. Thus, a leaf QT node is the root node for a multitype tree, and a leaf QT node can have a multitype tree depth (mttDepth) value of 0. If the multitype tree depth reaches MaxMttdepth (e.g., 4), further additional partitioning may not be considered. If the width of a multitype tree node is the same as MinBtSize and equal to or less than 2xMinTtSize, further additional horizontal partitioning may not be considered. If the height of a multitype tree node is the same as MinBtSize and equal to or less than 2xMinTtSize, further additional vertical partitioning may not be considered. When partitioning is not considered in this way, the encoding device can omit signaling of partitioning information. In such cases, the decoding device can induce the partitioning information to a predetermined value.

[0102] On the other hand, a single CTU can include a coding block for a luma sample (hereinafter referred to as a "luma block") and two coding blocks for corresponding chroma samples (hereinafter referred to as "chroma blocks"). The coding tree scheme described above can be applied similarly to the luma blocks and chroma blocks of a CU, or it can be applied separately. Specifically, luma blocks and chroma blocks within a single CTU can be divided into the same block tree structure, in which case the tree structure can be represented as a single tree (SINGLE_TREE). Alternatively, luma blocks and chroma blocks within a single CTU can be divided into separate block tree structures, in which case the tree structure can be represented as a dual tree (DUAL_TREE). In other words, when a CTU is divided into a dual tree, the block tree structure for luma blocks and the block tree structure for chroma blocks can exist separately. In this case, the block tree structure for a luma block can be called a dual-tree luma (DUAL_TREE_LUMA), and the block tree structure for a chroma block can be called a dual-tree chroma (DUAL_TREE_CHROMA). For P and B slice / tile groups, luma blocks and chroma blocks within a single CTU can be restricted to having the same coding tree structure. However, for I slice / tile groups, luma blocks and chroma blocks can have separate block tree structures from each other. If separate block tree structures are applied, a luma CTB (Coding Tree Block) can be divided into CUs based on a specific coding tree structure, and a chroma CTB can be divided into chroma CUs based on a different coding tree structure. That is, a CU within an I slice / tile group to which a separate block tree structure is applied can consist of a coding block for a luma component or a coding block for two chroma components, while a CU in a P or B slice / tile group can consist of a block for three color components (a luma component and two chroma components).

[0103] In the above, a quadtree coding tree structure with a multitype tree was described, but the structure in which a CU is split is not limited to this. For example, BT structures and TT structures can be interpreted as concepts included in multiple partitioning tree (MPT) structures, and a CU can be interpreted as being split by QT structures and MPT structures. In one example in which a CU is split by QT and MPT structures, the split structure can be determined by signaling a syntax element (e.g., MPT_split_type) containing information about how the leaf nodes of the QT structure are split into several blocks, and a syntax element (e.g., MPT_split_mode) containing information about whether the leaf nodes of the QT structure are split in the vertical or horizontal direction.

[0104] In another example, the CU can be divided in a way different from the QT, BT, or TT structures. That is, unlike the QT structure which divides the lower-depth CU into quarters the size of the upper-depth CU, or the BT structure which divides the lower-depth CU into half the size of the upper-depth CU, or the TT structure which divides the lower-depth CU into quarters or half the size of the upper-depth CU, the lower-depth CU can, depending on the case, be divided into 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 the size of the upper-depth CU, and the way in which the CU is divided is not limited to this.

[0105] Thus, the quadtree coding block structure with the multitype tree can provide a highly flexible block partition structure. On the other hand, due to the partition types supported by the multitype tree, different partition patterns may, in some cases, lead to potentially identical coding block structures. By limiting the occurrence of such redundant partition patterns, the encoding and decoding devices can reduce the amount of data in the partition information.

[0106] For example, Figure 8 illustrates redundant partition patterns that can occur in binary and ternary tree partitions. As shown in Figure 8, a 2-step level unidirectional consecutive binary partition 810 and 820 has the same coding block structure as a binary partition on the center partition after a ternary partition. In such a case, a binary tree partition on the center blocks 830 and 840 of the ternary partition can be prohibited. Such prohibitions can be applied to the CU of all pictures. When such a particular partition is prohibited, the signaling of the corresponding syntax element can be modified to reflect this prohibition, thereby reducing the number of bits signaled for the partition. For example, if a binary tree partition on the center block of a CU is prohibited, as in the example shown in Figure 8, the mtt_split_cu_binary_flag syntax element, which indicates whether the partition is a binary or ternary partition, is not signaled, and its value can be induced to 0 by the decoder.

[0107] Overview of Interpretation

[0108] The following explains the inter-prediction based on this disclosure.

[0109] The prediction unit of the image encoding / decoding device according to this disclosure can perform interpretation on a block-by-block basis to derive predicted samples. Interpretation can indicate a prediction derived in a manner dependent on data elements of pictures other than the current picture (e.g., sample values ​​or motion information). When interpretation is applied to the current block, a predicted block (predicted block or predicted sample array) for the current block can be derived based on a reference block (reference sample array) identified by motion vectors on the reference picture pointed to by the reference picture index. At this time, in order to reduce the amount of motion information transmitted in interpretation mode, the motion information of the current block can be predicted on a block, subblock, or sample basis based on the correlation of motion information between the surrounding blocks and the current block. The motion information may include motion vectors and reference picture indexes. The motion information may further include interpretation type information (L0 prediction, L1 prediction, Bi prediction, etc.). When interpretation is applied, the surrounding blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture containing the reference block and the reference picture containing the time-peripheral block may be the same or different. The time-peripheral block may be called by names such as collocated reference block, colCU, or colBlock, and the reference picture containing the time-peripheral block may be called by names such as collocated picture (colPic) or colPicture. For example, a list of motion information candidates may be constructed based on the surrounding blocks of the current block, and flags or index information indicating which candidate is selected (used) may be signaled to derive the motion vector and / or reference picture index of the current block.

[0110] Interpretation can be performed based on various prediction modes. For example, in skip mode and merge mode, the motion information of the current block may be identical to the motion information of the selected surrounding block. In skip mode, unlike merge mode, residual signals may not be transmitted. In motion vector prediction (MVP) mode, the motion vector of the selected surrounding block can be used as a motion vector predictor, and the motion vector difference can be signaled. In this case, the motion vector of the current block can be derived using the sum of the motion vector predictor and the motion vector difference. In this disclosure, MVP mode can be used interchangeably with AMVP (Advanced Motion Vector Prediction).

[0111] The motion information may include L0 motion information and / or L1 motion information based on the interpretation type (L0 prediction, L1 prediction, Bi prediction, etc.). A motion vector in the L0 direction may be called an L0 motion vector or MVL0, and a motion vector in the L1 direction may be called an L1 motion vector or MVL1. A prediction based on the L0 motion vector may be called an L0 prediction, a prediction based on the L1 motion vector may be called an L1 prediction, and a prediction based on both the L0 motion vector and the L1 motion vector may be called a bi (Bi) prediction. Here, the L0 motion vector may represent a motion vector associated with the reference picture list L0 (L0), and the L1 motion vector may represent a motion vector associated with the reference picture list L1 (L1). The reference picture list L0 may include pictures earlier in the output order than the current picture as reference pictures, and the reference picture list L1 may include pictures later in the output order than the current picture. The aforementioned earlier pictures can be called forward (reference) pictures, and the aforementioned later pictures can be called backward (reference) pictures. The reference picture list L0 may further include pictures that are later in the output order than the current picture as reference pictures. In this case, the earlier picture may be indexed first in the reference picture list L0, and the later picture may be indexed next. The reference picture list L1 may further include pictures that are earlier in the output order than the current picture as reference pictures. In this case, the later picture may be indexed first in the reference picture list L1, and the earlier picture may be indexed next. Here, the output order may correspond to the POC (picture order count) order.

[0112] Figure 9 is a flowchart showing an interprediction-based video / image coding method.

[0113] Figure 10 is a diagram illustrating the configuration of the interpretation unit 180 according to this disclosure.

[0114] The encoding method in Figure 9 can be performed by the image encoding device in Figure 2. Specifically, step S610 can be performed by the interprediction unit 180, and step S620 can be performed by the residual processing unit. Specifically, step S620 can be performed by the subtraction unit 115. Step S630 can be performed by the entropy encoding unit 190. The prediction information in step S630 is derived by the interprediction unit 180, and the residual information in step S630 can be derived by the residual processing unit. The residual information is information about the residual sample. The residual information may include information about the quantized conversion coefficients for the residual sample. As described above, the residual sample is derived as a conversion coefficient via the conversion unit 120 of the image encoding device, and the conversion coefficient can be derived as a quantized conversion coefficient via the quantization unit 130. Information about the quantized conversion coefficients can be encoded by the entropy encoding unit 190 via the residual coding procedure.

[0115] The image coding device can perform inter prediction for the current block (S610). The image coding device can derive the inter prediction mode and motion information of the current block and generate a prediction sample for the current block. Here, the inter prediction mode determination, motion information derivation, and prediction sample generation procedures may be performed simultaneously, or one procedure may be performed before the others. For example, as shown in Figure 10, the inter prediction unit 180 of the image coding device may include a prediction mode determination unit 181, a motion information derivation unit 182, and a prediction sample derivation unit 183. The prediction mode determination unit 181 can determine the prediction mode for the current block, the motion information derivation unit 182 can derive the motion information of the current block, and the prediction sample derivation unit 183 can derive a prediction sample for the current block. For example, the inter prediction unit 180 of the image coding device can search for blocks similar to the current block within a certain area (search area) of the reference picture via motion estimation and derive a reference block whose difference from the current block is the minimum or below a certain standard. Based on this, a reference picture index pointing to the reference picture in which the reference block is located can be derived, and a motion vector can be derived based on the positional difference between the reference block and the current block. The image encoding device can determine which of the various prediction modes is to be applied to the current block. The image encoding device can compare the rate-distortion (RD) cost for the various prediction modes and determine the optimal prediction mode for the current block. However, the method by which the image encoding device determines the prediction mode for the current block is not limited to the above example, and various methods can be used.

[0116] For example, when skip mode or merge mode is applied to the current block, the image encoding device can derive merge candidates from surrounding blocks of the current block and construct a merge candidate list using the deriveted merge candidates. The image encoding device can also derive a reference block from among the reference blocks pointed to by the merge candidates included in the merge candidate list whose difference from the current block is the minimum or below a certain standard. In this case, a merge candidate associated with the derived reference block is selected, and merge index information indicating the selected merge candidate is generated and signaled to the image decoding device. The motion information of the current block can be derived using the motion information of the selected merge candidate.

[0117] As another example, when the MVP mode is applied to the current block, the image encoding device can derive motion vector predictor (mvp) candidates from the surrounding blocks of the current block and construct an mvp candidate list using the deriveted mvp candidates. The image encoding device can also use the motion vector of a selected mvp candidate from among the mvp candidates included in the mvp candidate list as the mvp of the current block. In this case, for example, the motion vector pointing to the reference block derived by the motion estimation described above can be used as the motion vector of the current block, and the mvp candidate with the smallest difference between its motion vector and the motion vector of the current block can become the selected mvp candidate. The motion vector difference (MVD), which is the difference obtained by subtracting the mvp from the motion vector of the current block, can be derived. In this case, index information indicating the selected mvp candidate and information regarding the MVD can be signaled to the image decoding device. Furthermore, when MVP mode is applied, the value of the reference picture index can be composed of reference picture index information and separately signaled to the image decoding device.

[0118] The image encoding device can derive a residual sample based on the predicted sample (S620). The image encoding device can derive the residual sample by comparing the original sample of the current block with the predicted sample. For example, the residual sample can be derived by subtracting the corresponding predicted sample from the original sample.

[0119] The image encoding device can encode image information including prediction information and residual information (S630). The image encoding device can output the encoded image information in bitstream format. The prediction information is information related to the prediction procedure and may include prediction mode information (e.g., skip flag, merge flag, or mode index) and motion information. Of the prediction mode information, the skip flag indicates whether or not the skip mode is applied to the current block, and the merge flag indicates whether or not the merge mode is applied to the current block. Alternatively, the prediction mode information may be information that indicates one of several prediction modes, such as the mode index. If the skip flag and merge flag are both 0, it can be determined that the MVP mode is applied to the current block. The motion information may include candidate selection information (e.g., merge index, mvp flag, or mvp index) which is information for deriving a motion vector. Of the candidate selection information, the merge index may be signaled when the merge mode is applied to the current block and may be information for selecting one of the merge candidates included in the merge candidate list. Of the candidate selection information, the mvp flag or mvp index can be signaled when MVP mode is applied to the current block, and may be information for selecting one of the mvp candidates included in the mvp candidate list. The motion information may also include the MVD information and / or reference picture index information described above. The motion information may also include information indicating whether L0 prediction, L1 prediction, or bi (Bi) prediction is applied. The residual information is information about the residual sample. The residual information may also include information about the quantized transformation coefficients for the residual sample.

[0120] The output bitstream can be stored on a (digital) storage medium and transmitted to an image decoding device, or it can be transmitted to an image decoding device via a network.

[0121] On the other hand, as mentioned above, the image coding device can generate a reconstructed picture (a picture including a reconstructed sample and a reconstructed block) based on the reference sample and the residual sample. This is because the image coding device can derive the same prediction results as the image decoding device, thereby improving coding efficiency. Therefore, the image coding device can store the reconstructed picture (or reconstructed sample, reconstructed block) in memory and use it as a picture for interpretation. As mentioned above, in-loop filtering procedures and the like can be further applied to the reconstructed picture.

[0122] Figure 11 is a flowchart of an interpretation-based video / image decoding method.

[0123] Figure 12 is a diagram illustrating the configuration of the interpretation unit 260 according to this disclosure.

[0124] The image decoding device can perform operations corresponding to those performed by the image encoding device. The image decoding device can make predictions for the current block based on the received prediction information and derive prediction samples.

[0125] The decoding method in Figure 11 can be performed by the image decoding device in Figure 3. Steps S810 to S830 can be performed by the interpretation unit 260, and the prediction information in step S810 and the residual information in step S840 can be obtained from the bitstream by the entropy decoding unit 210. The residual processing unit of the image decoding device can derive a residual sample for the current block based on the residual information (S840). Specifically, the inverse quantization unit 220 of the residual processing unit derives a conversion coefficient by performing inverse quantization based on the quantized conversion coefficient derived from the residual information, and the inverse transformation unit 230 of the residual processing unit can derive a residual sample for the current block by performing an inverse transformation on the conversion coefficient. Step S850 can be performed by the addition unit 235 or the reconstruction unit.

[0126] Specifically, the image decoding device can determine the prediction mode for the current block based on the received prediction information (S810). Based on the prediction mode information in the prediction information, the image decoding device can determine which interpretation mode is applied to the current block.

[0127] For example, based on the skip flag, it can be determined whether the skip mode is applied to the current block. Alternatively, based on the merge flag, it can be determined whether the merge mode is applied to the current block or whether the MVP mode is determined. Or, based on the mode index, one of several inter-prediction mode candidates can be selected. The inter-prediction mode candidates may include the skip mode, merge mode, and / or the MVP mode, or may include various inter-prediction modes as described later.

[0128] The image decoding device can derive motion information for the current block based on the determined interprediction mode (S820). For example, if a skip mode or merge mode is applied to the current block, the image decoding device can configure a merge candidate list, which will be described later, and select one of the merge candidates included in the merge candidate list. This selection can be made based on the candidate selection information (merge index) described above. The motion information for the current block can be derived using the motion information for the selected merge candidate. For example, the motion information for the selected merge candidate can be used as the motion information for the current block.

[0129] As another example, when the MVP mode is applied to the current block, the image decoding device can configure an MVP candidate list and use the motion vector of an MVP candidate selected from the MVP candidates included in the MVP candidate list as the MVP of the current block. The selection can be made based on the candidate selection information (MVP flag or MVP index) described above. In this case, the MVD of the current block can be derived based on the information regarding the MVD, and the motion vector of the current block can be derived based on the MVP of the current block and the MVD. Furthermore, the reference picture index of the current block can be derived based on the reference picture index information. The picture pointed to by the reference picture index in the associated reference picture list for the current block can be derived as the reference picture referenced for interpretation of the current block.

[0130] The image decoding device can generate predicted samples for the current block based on the motion information of the current block (S830). In this case, the reference picture can be derived based on the reference picture index of the current block, and the predicted samples for the current block can be derived using the sample of the reference block pointed to by the motion vector of the current block on the reference picture. Depending on the case, a prediction sample filtering procedure can be further performed on all or some of the predicted samples for the current block.

[0131] For example, as shown in Figure 12, the interpretation unit 260 of the image decoding device may include a prediction mode determination unit 261, a motion information derivation unit 262, and a prediction sample derivation unit 263. The interpretation unit 260 of the image decoding device can determine a prediction mode for the current block based on prediction mode information received from the prediction mode determination unit 261, derive motion information (such as motion vectors and / or reference picture indices) for the current block based on motion information received from the motion information derivation unit 262, and derive prediction samples for the current block using the prediction sample derivation unit 263.

[0132] The image decoding device can generate a residual sample for the current block based on the received residual information (S840). The image decoding device can generate a reconstructed sample for the current block based on the predicted sample and the residual sample, and generate a reconstructed picture based on this (S850). As previously mentioned, further procedures such as in-loop filtering can be applied to the reconstructed picture thereafter.

[0133] As described above, the interpretation procedure may include an interpretation mode determination step, a motion information derivation step based on the determined prediction mode, and a prediction execution step (generation of prediction samples) based on the derived motion information. The interpretation procedure can be performed by an image encoding device and an image decoding device as described above.

[0134] The following describes in more detail the steps for deriving motion information using the prediction mode.

[0135] As mentioned above, interpretation can be performed using motion information of the current block. The image encoding device can derive optimal motion information for the current block through a motion estimation procedure. For example, the image encoding device can use the original block in the original picture to search for a highly correlated similar reference block in fractional pixel units within a defined search range in the reference picture, thereby deriving motion information. Block similarity can be calculated based on the sum of absolute differences (SAD) between the current block and the reference block. In this case, motion information can be derived based on the reference block with the smallest SAD within the search area. The derived motion information can be signaled to the image decoding device in various ways based on the interpretation mode.

[0136] When merge mode is applied to a current block, the movement information of the current block is not transmitted directly, but rather the movement information of the current block is guided using the movement information of surrounding blocks. Therefore, the movement information of the current predicted block can be instructed by transmitting flag information indicating that merge mode has been used and candidate selection information (e.g., merge index) indicating which surrounding blocks were used as merge candidates. In this disclosure, since the current block is the unit of prediction execution, the current block can be used in the same sense as the current predicted block, and the surrounding block can be used in the same sense as the surrounding predicted block.

[0137] The image encoding device can search for merge candidate blocks to be used to guide the motion information of the current block in order to perform a merge mode. For example, up to five merge candidate blocks may be used, but this is not limited to them. The maximum number of merge candidate blocks may be transmitted from the slice header or tile group header, but this is not limited to them. After finding the merge candidate blocks, the image encoding device can generate a merge candidate list and select the merge candidate block with the lowest RD cost from among them as the final merge candidate block.

[0138] This disclosure provides various embodiments of merge candidate blocks that constitute the merge candidate list. The merge candidate list may use, for example, five merge candidate blocks. For example, it may use four spatial merge candidates and one temporal merge candidate.

[0139] Figure 13 illustrates surrounding blocks that can be used as candidates for spatial merge.

[0140] Figure 14 is a schematic diagram illustrating a method for constructing a merge candidate list according to an example of this disclosure.

[0141] The image encoding / decoding device can search for spatially surrounding blocks of the current block and insert the derived spatial merge candidates into the merge candidate list (S1110). For example, the spatially surrounding blocks may include the left lower corner surrounding block A0, the left surrounding block A1, the right upper corner surrounding block B0, the upper surrounding block B1, and the left upper corner surrounding block B2 of the current block, as shown in Figure 13. However, this is merely an example, and additional surrounding blocks such as the right surrounding block, the lower surrounding block, and the right lower surrounding block can also be used as spatially surrounding blocks. The image encoding / decoding device can detect available blocks by searching for the spatially surrounding blocks based on priority and derive the movement information of the detected blocks as spatial merge candidates. For example, the image encoding / decoding device can construct a merge candidate list by searching the five blocks shown in Figure 13 in the order A1, B1, B0, A0, B2 and sequentially indexing the available candidates.

[0142] The image encoding / decoding device can search for time-peripheral blocks of the current block and insert the derived time merge candidates into the merge candidate list (S1120). The time-peripheral blocks can be located on a reference picture which is a different picture from the current picture on which the current block is located. The reference picture on which the time-peripheral blocks are located can be called a collocated picture or a col picture. The time-peripheral blocks can be searched on the col picture in the order of the lower right corner block and the lower right center block of the colocated block relative to the current block. On the other hand, when motion data compression is applied to reduce memory load, specific motion information can be stored as representative motion information for each certain storage unit in the col picture. In this case, it is not necessary to store motion information for all blocks in the certain storage unit, thereby achieving the motion data compression effect. In this case, the certain storage unit can be predetermined, for example, a 16x16 sample unit or an 8x8 sample unit, or size information for the certain storage unit can be signaled from the image encoding device to the image decoding device. When the motion data compression described above is applied, the motion information of the time-period block can be replaced with representative motion information of the constant storage unit in which the time-period block is located. In other words, in this case, from an implementation standpoint, instead of a predicted block lock located at the coordinates of the time-period block, the time merge candidate can be derived based on the motion information of the predicted block that covers the position after being arithmetically shifted to the right by a certain value based on the coordinates of the time-period block (top-left sample position) and then arithmetically shifted to the left. For example, if the constant storage unit is 2 n ×2 nWhen it is in the sample unit, if the coordinates of the time neighboring block are (xTnb, yTnb), the motion information of the prediction block located at the corrected position ((xTnb>>n)<<n), (yTnb>>n)<<n)) can be used for the time merge candidate. Specifically, for example, when the fixed storage unit is 16×16 sample units, if the coordinates of the time neighboring block are (xTnb, yTnb), the motion information of the prediction block located at the corrected position ((xTnb>>4)<<4), (yTnb>>4)<<4)) can be used for the time merge candidate. Or, for example, when the fixed storage unit is 8×8 sample units, if the coordinates of the time neighboring block are (xTnb, yTnb), the motion information of the prediction block located at the corrected position ((xTnb>>3)<<3), (yTnb>>3)<<3)) can be used for the time merge candidate.

[0143] Referring to FIG. 14 again, the image encoding device / image decoding device can check whether the number of current merge candidates is smaller than the number of maximum merge candidates (S1130). The number of the maximum merge candidates can be defined in advance or signaled from the image encoding device to the image decoding device. For example, the image encoding device can generate information regarding the number of the maximum merge candidates, encode it, and transmit it to the image decoding device in the form of a bit stream. When all of the number of the maximum merge candidates are satisfied, the subsequent candidate addition process (S1140) can be not performed.

[0144] If the check result in step S1130 is that the number of the current merge candidates is smaller than the number of the maximum merge candidates, the image encoding device / image decoding device can induce additional merge candidates based on a predetermined method and then insert them into the merge candidate list (S1140).

[0145] If, as a result of the verification in step S1130, the number of current merge candidates is not less than the number of maximum merge candidates, the image encoding / decoding device can terminate the configuration of the merge candidate list. In this case, the image encoding device can select the optimal merge candidate from among the merge candidates constituting the merge candidate list based on the RD cost, and can signal candidate selection information (e.g., merge index) pointing to the selected merge candidate to the image decoding device. The image decoding device can select the optimal merge candidate based on the merge candidate list and the candidate selection information.

[0146] As described above, the motion information of the selected merge candidate can be used as the motion information of the current block, and predicted samples of the current block can be derived based on the motion information of the current block. The image encoding device can derive the residual samples of the current block based on the predicted samples and signal the image decoding device to the residual information of the residual samples. As described above, the image decoding device can generate restored samples based on the residual samples derived based on the residual information and the predicted samples, and generate a restored picture based on these.

[0147] When skip mode is applied to a block, the motion information of the current block can be derived in the same way as when merge mode was applied previously. However, when skip mode is applied, the residual signal for that block is omitted. Therefore, the predicted sample can be immediately used as the reconstructed sample.

[0148] When MVP mode is applied to the current block, a motion vector predictor (MVP) candidate list can be generated using the motion vectors of the restored spatial surrounding blocks (e.g., the surrounding blocks shown in Figure 13) and / or the motion vectors corresponding to the temporal surrounding blocks (or Col blocks). In other words, the motion vectors of the restored spatial surrounding blocks and / or the motion vectors corresponding to the temporal surrounding blocks can be used as motion vector predictor candidates for the current block. When dual prediction is applied, an MVP candidate list for L0 motion information derivation and an MVP candidate list for L1 motion information derivation are generated and available separately. The prediction information (or information related to prediction) for the current block may include candidate selection information (e.g., an MVP flag or MVP index) that indicates the optimal motion vector predictor candidate selected from among the motion vector predictor candidates included in the MVP candidate list. In this case, the prediction unit can use the candidate selection information to select the motion vector predictor for the current block from among the motion vector predictor candidates included in the MVP candidate list. The prediction unit of the image encoding device can calculate the motion vector difference (MVD) between the motion vector of the current block and the motion vector predictor, and can encode this and output it in bitstream format. In other words, the MVD can be calculated by subtracting the motion vector predictor from the motion vector of the current block. The prediction unit of the image decoding device can obtain the motion vector difference contained in the prediction information and derive the motion vector of the current block by adding the motion vector difference and the motion vector predictor. The prediction unit of the image decoding device can obtain or derive a reference picture index that indicates a reference picture from the prediction information.

[0149] Figure 15 is a schematic diagram illustrating a method for constructing a motion vector predictor candidate list according to an example of this disclosure.

[0150] First, the system searches for spatial candidate blocks in the current block and inserts any available candidate blocks into the MVP candidate list (S1210). Then, it is determined whether there are fewer than two MVP candidates in the MVP candidate list (S1220). If there are two, the construction of the MVP candidate list can be completed.

[0151] In step S1220, if there are fewer than two available spatial candidate blocks, the system can search for a time candidate block for the current block and insert any available candidate blocks into the MVP candidate list (S1230). If no time candidate blocks are available, the system can complete the construction of the MVP candidate list by inserting a zero motion vector into the MVP candidate list (S1240).

[0152] On the other hand, when MVP mode is applied, the reference picture index can be explicitly signaled. In this case, the picture index for L0 prediction (refidxL0) and the reference picture index for L1 prediction (refidxL1) can be separately signaled. For example, when MVP mode is applied and bi-indicator prediction (BI prediction) is applied, both the information regarding refidxL0 and the information regarding refidxL1 can be signaled.

[0153] As mentioned above, when MVP mode is applied, information about the MVD derived from the image encoding device can be signaled to the image decoding device. The information about the MVD may include, for example, information indicating the x and y components for the MVD absolute value and sign. In this case, information indicating whether the MVD absolute value is greater than 0, whether it is greater than 1, and the rest of the MVD can be signaled stepwise. For example, information indicating whether the MVD absolute value is greater than 1 can be signaled only if the value of the flag information indicating whether the MVD absolute value is greater than 0 is 1.

[0154] Figure 16 shows a syntax structure for transmitting an MVD from an image encoding device to an image decoding device according to an example of this disclosure.

[0155] In Figure 16, abs_mvd_greater0_flag[0] indicates whether the absolute value of the x component of the MVD is greater than 0, and abs_mvd_greater0_flag[1] indicates whether the absolute value of the y component of the MVD is greater than 0. Similarly, abs_mvd_greater1_flag[0] indicates whether the absolute value of the x component of the MVD is greater than 1, and abs_mvd_greater1_flag[1] indicates whether the absolute value of the y component of the MVD is greater than 1. As shown in Figure 16, abs_mvd_greater1_flag can only be transmitted when abs_mvd_greater0_flag is 1. In Figure 16, abs_mvd_minus2 indicates the absolute value of the MVD minus 2, and mvd_sign_flag indicates whether the sign of the MVD is positive or negative. Using the syntax structure shown in Figure 16, MVD can be derived as follows:

[0156] [Mathematics 1]

[0157] MVD[compIdx]=abs_mvd_greater0_flag[compIdx]*abs_mvd_minus2[compIdx]+2*1-2*mvd_sign_flag[compIdx]

[0158] On the other hand, MVDs for L0 prediction (MVDL0) and MVDs for L1 prediction (MVDL1) can be signaled separately, and the information regarding the MVDs may include information regarding MVDL0 and / or information regarding MVDL1. For example, if MVP mode is applied to the current block and BI prediction is applied, both the information regarding MVDL0 and the information regarding MVDL1 can be signaled.

[0159] Overview of IBC (Intra Block Copy) Prediction

[0160] The IBC forecast based on this disclosure is explained below.

[0161] IBC prediction can be performed in the prediction unit of an image encoding / decoding device. IBC prediction can be simply referred to as "IBC". The IBC can be used for content image / video coding such as games, for example, as in SCC (screen content coding). The IBC basically performs prediction within the current picture, but can be performed similarly to interpretation in that it derives a reference block within the current picture. In other words, the IBC can use at least one of the interpretation techniques described in this disclosure. For example, the IBC can use at least one of the motion information (motion vector) derivation methods described above. At least one of the interpretation techniques can also be used with some modification taking into consideration the IBC prediction. The IBC can reference the current picture. Therefore, it can also be called CPR (current picture referencing).

[0162] For IBC, the image encoding device can perform block matching BM to derive the optimal block vector (or motion vector) for the current block (e.g., CU). The derived block vector (or motion vector) can be signaled to the image decoding device via a bitstream using a method similar to the motion information (motion vector) signaling in interpretation described above. The image decoding device can derive a reference block for the current block in the current picture via the signaled block vector (motion vector), thereby deriving a prediction signal (predicted block or predicted sample) for the current block. Here, the block vector (or motion vector) can indicate the displacement from the current block to the reference block located in an already restored region within the current picture. Therefore, the block vector (or motion vector) can also be called a displacement vector. Hereinafter, the motion vector in IBC can correspond to the block vector or the displacement vector. The motion vector of the current block can include a motion vector for the luma component (luma motion vector) or a motion vector for the chroma component (chroma motion vector). For example, the chroma motion vector for an IBC-coded CU can also be integer-sample-level (i.e., integer precision). The chroma motion vector can also be clipped to integer-sample levels. As mentioned above, IBC can employ at least one of the interpretation techniques, and for example, the chroma motion vector can be encoded / decoded using the merge mode or MVP mode described above.

[0163] When merge mode is applied to a Luma IBC block, the merge candidate list for the Luma IBC block can be constructed in the same way as the merge candidate list in intermode, as described with reference to Figure 14. However, in the case of Luma IBC blocks, time-peripheral blocks do not need to be used as merge candidates.

[0164] When MVP mode is applied to a Luma IBC block, the MVP candidate list for the Luma IBC block can be configured in the same way as the MVP candidate list in inter-mode, as described with reference to Figure 15. However, in the case of Luma IBC blocks, time candidate blocks do not necessarily have to be used as MVP candidates.

[0165] The IBC derives the reference block from the already restored area within the current picture. To reduce memory consumption and the complexity of the image decoding device, only predefined areas within the already restored area of ​​the current picture can be referenced. These predefined areas may include the current CTU containing the current block. By restricting the referenced restored area to the predefined area in this way, the IBC mode can be implemented in hardware using local on-chip memory.

[0166] An image encoding device performing IBC can search the previously defined region to determine the reference block with the smallest RD cost, and derive a motion vector (block vector) based on the positions of the reference block and the current block.

[0167] Whether or not to apply IBC to the current block can be signaled at the CU level as IBC execution information. Information regarding the signaling method for the current block's motion vector (IBC MVP mode or IBC skip / merge mode) can also be signaled. IBC execution information can be used to determine the prediction mode of the current block. Therefore, IBC execution information can be included in the information regarding the prediction mode of the current block.

[0168] In IBC skip / merge mode, the merge candidate index is signaled and can be used to indicate which block vector is currently used to predict the luma block from among the block vectors included in the merge candidate list. In this case, the merge candidate list can include peripheral blocks encoded by IBC. The merge candidate list can be configured to include spatial merge candidates but not temporal merge candidates. Furthermore, the merge candidate list can include HMVP (History-based motion vector predictor) candidates and / or pairwise candidates.

[0169] In IBC MVP mode, block vector difference values ​​can be encoded in the same way as the motion vector difference values ​​in intermode described above. The block vector prediction method, similar to intermode's MVP mode, can be constructed and used by configuring an MVP candidate list containing two candidates as predictors. One of the two candidates can be derived from the left peripheral block, and the other from the upper peripheral block. In this case, a candidate can only be derived from the peripheral block if the left or upper peripheral block is encoded with IBC. If the left or upper peripheral block is not available, for example, if it is not encoded with IBC, a default block vector can be included in the MVP candidate list as a predictor. Also, similar to intermode's MVP mode, information (e.g., a flag) to indicate one of the two block vector predictors is signaled and used as candidate selection information. The MVP candidate list can include an HMVP candidate and / or a zero motion vector as the default block vector.

[0170] The aforementioned HMVP candidates are sometimes called history-based MVP candidates. MVP candidates, merge candidates, or block vector candidates previously used in the encoding / decoding of the current block can be stored in the HMVP list as HMVP candidates. Thereafter, if the current block's merge candidate list or MVP candidate list does not contain the maximum number of candidates, candidates stored in the HMVP list can be added to the current block's merge candidate list or MVP candidate list as HMVP candidates.

[0171] The pairwise candidates mentioned above refer to candidates that are derived by selecting two candidates in a predetermined order from among the candidates already included in the current block merge candidate list, and then averaging the two selected candidates.

[0172] Figure 17 is a flowchart of an IBC-based video / image encoding method.

[0173] Figure 18 is a diagram illustrating the configuration of a prediction unit that implements the IBC-based video / image coding method according to this disclosure.

[0174] The encoding method shown in Figure 17 can be performed by the image encoding device shown in Figure 2. Specifically, step S1410 can be performed by the prediction unit, and step S1420 can be performed by the residual processing unit. Specifically, step S1420 can be performed by the subtraction unit 115. Step S1430 can be performed by the entropy encoding unit 190. The prediction information in step S1430 is derived by the prediction unit, and the residual information in step S1430 can be derived by the residual processing unit. The residual information is information about the residual sample. The residual information may include information about the quantized conversion coefficients for the residual sample. As described above, the residual sample is derived as a conversion coefficient via the conversion unit 120 of the image encoding device, and the conversion coefficient can be derived as a quantized conversion coefficient via the quantization unit 130. Information about the quantized conversion coefficients can be encoded by the entropy encoding unit 190 via the residual coding procedure.

[0175] The image coding device can perform IBC prediction (IBC-based prediction) for the current block (S1410). The image coding device can derive the prediction mode and motion vector (block vector) of the current block and generate prediction samples for the current block. The prediction mode can include at least one of the inter-prediction modes described above. Here, the steps of prediction mode determination, motion vector derivation, and prediction sample generation may be performed simultaneously, or any one of the steps may be performed first. For example, as shown in Figure 18, the prediction unit of an image coding device performing an IBC-based video / image coding method can include a prediction mode determination unit, a motion vector derivation unit, and a prediction sample derivation unit. The prediction mode determination unit can determine the prediction mode for the current block, the motion vector derivation unit can derive the motion vector of the current block, and the prediction sample derivation unit can derive prediction samples for the current block. For example, the prediction unit of the image encoding device can search for blocks similar to the current block within the restored area of ​​the current picture (or a certain area within the restored area (search area)) and derive a reference block whose difference from the current block is the minimum or below a certain standard. The image encoding device can derive a motion vector based on the displacement difference between the reference block and the current block. The image encoding device can determine which of various prediction modes to apply to the current block. The image encoding device can compare the rate distortion costs (RD costs) for the various prediction modes and determine the optimal prediction mode for the current block. However, the method by which the image encoding device determines the prediction mode for the current block is not limited to the above example, and various methods can be used.

[0176] For example, when skip mode or merge mode is applied to the current block, the image encoding device can derive merge candidates from surrounding blocks of the current block and construct a merge candidate list using the deriveted merge candidates. The image encoding device can also derive a reference block from among the reference blocks pointed to by the merge candidates included in the merge candidate list whose difference from the current block is the minimum or below a certain standard. In this case, a merge candidate associated with the derived reference block is selected, and merge index information pointing to the selected merge candidate is generated and signaled to the image decoding device. The motion vector of the current block can be derived using the motion vector of the selected merge candidate.

[0177] As another example, when the MVP mode is applied to the current block, the image encoding device can derive motion vector predictor (mvp) candidates from the surrounding blocks of the current block and construct an mvp candidate list using the deriveted mvp candidates. The image encoding device can also use the motion vector of an mvp candidate selected from the mvp candidates included in the mvp candidate list as the mvp of the current block. In this case, for example, the motion vector pointing to the reference block derived by the motion estimation described above can be used as the motion vector of the current block, and the mvp candidate having the smallest difference between its motion vector and the motion vector of the current block can become the selected mvp candidate. The motion vector difference (MVD), which is the difference obtained by subtracting the mvp from the motion vector of the current block, can be derived. In this case, index information pointing to the selected mvp candidate and information regarding the MVD can be signaled to the image decoding device.

[0178] The image encoding device can derive a residual sample based on the predicted sample (S1420). The image encoding device can derive the residual sample by comparing the original sample of the current block with the predicted sample. For example, the residual sample can be derived by subtracting the corresponding predicted sample from the original sample.

[0179] The image encoding device can encode image information including prediction information and residual information (S1430). The image encoding device can output the encoded image information in bitstream format. The prediction information may include prediction mode information (e.g., skip flag, merge flag, or mode index) and motion vector information as information related to the prediction procedure. Of the prediction mode information, the skip flag indicates whether or not the skip mode is applied to the current block, and the merge flag indicates whether or not the merge mode is applied to the current block. Alternatively, the prediction mode information may be information that indicates one of several prediction modes, such as the mode index. If the skip flag and merge flag are both 0, it can be determined that the MVP mode is applied to the current block. The motion vector information may include candidate selection information (e.g., merge index, mvp flag, or mvp index) which is information for deriving the motion vector. Of the candidate selection information, the merge index may be signaled when the merge mode is applied to the current block, and may be information for selecting one of the merge candidates included in the merge candidate list. The MVP flag or MVP index among the candidate selection information can be signaled when the MVP mode is applied to the current block, and can be information for selecting one of the MVP candidates included in the MVP candidate list. The motion vector information may also include the MVD information described above. The motion vector information may also include information indicating whether L0 prediction, L1 prediction, or bi prediction is applied. The residual information is information about the residual sample. The residual information may include information about the quantized transformation coefficients for the residual sample.

[0180] The output bitstream can be stored on a (digital) storage medium and transmitted to an image decoding device, or it can be transmitted to an image decoding device via a network.

[0181] On the other hand, as mentioned above, the image coding device can generate a reconstructed picture (a picture including the reconstructed sample and the reconstructed block) based on the reference sample and the residual sample. This is because the image coding device derives the same prediction results as the image decoding device, thereby improving coding efficiency. Therefore, the image coding device can store the reconstructed picture (or reconstructed sample, reconstructed block) in memory and use it as a reference picture for interpretation. As mentioned above, in-loop filtering procedures and the like can be further applied to the reconstructed picture.

[0182] Figure 19 is a flowchart showing an IBC-based video / image decoding method.

[0183] Figure 20 is an illustrative diagram showing the configuration of a prediction unit that performs the IBC-based video / image decoding method according to this disclosure.

[0184] The image decoding device can perform operations corresponding to those performed by the image encoding device. Based on the received prediction information, the image decoding device can perform IBC prediction for the current block and derive prediction samples.

[0185] The decoding method in Figure 19 can be performed by the image decoding device in Figure 3. Steps S1610 to S1630 can be performed by the prediction unit, and the prediction information in step S1610 and the residual information in step S1640 can be obtained from the bitstream by the entropy decoding unit 210. The residual processing unit of the image decoding device can derive a residual sample for the current block based on the residual information (S1640). Specifically, the inverse quantization unit 220 of the residual processing unit derives a conversion coefficient by performing inverse quantization based on the quantized conversion coefficient derived based on the residual information, and the inverse transformation unit 230 of the residual processing unit can derive a residual sample for the current block by performing an inverse transformation on the conversion coefficient. Step S1650 can be performed by the addition unit 235 or the reconstruction unit.

[0186] Specifically, the image decoding device can determine the prediction mode for the current block based on the received prediction information (S1610). Based on the prediction mode information in the prediction information, the image decoding device can determine which prediction mode is applied to the current block.

[0187] For example, based on the skip flag, it can be determined whether the skip mode is applied to the current block. Alternatively, based on the merge flag, it can be determined whether the merge mode is applied to the current block or whether the MVP mode is determined. Or, based on the mode index, one of the various prediction mode candidates can be selected. The prediction mode candidates may include the skip mode, merge mode and / or the MVP mode, or may include the various interpretation modes described above.

[0188] The image decoding device can derive the motion vector of the current block based on the determined prediction mode (S1620). For example, if a skip mode or merge mode is applied to the current block, the image decoding device can configure the merge candidate list described above and select one of the merge candidates included in the merge candidate list. The selection can be made based on the candidate selection information (merge index) described above. The motion vector of the current block can be derived using the motion vector of the selected merge candidate. For example, the motion vector of the selected merge candidate can be used as the motion vector of the current block.

[0189] As another example, when the MVP mode is applied to the current block, the image decoding device can configure an MVP candidate list and use the motion vector of an MVP candidate selected from the MVP candidates included in the MVP candidate list as the MVP of the current block. The selection can be made based on the candidate selection information (MVP flag or MVP index) described above. In this case, the MVD of the current block can be derived based on the information regarding the MVD, and the motion vector of the current block can be derived based on the MVP of the current block and the MVD.

[0190] The image decoding device can generate predicted samples for the current block based on the motion vector of the current block (S1630). The predicted samples for the current block can be derived using the samples of the reference block pointed to by the motion vector of the current block on the current picture. Depending on the case, a prediction sample filtering procedure may be further performed on all or some of the predicted samples for the current block.

[0191] For example, as shown in Figure 20, the prediction unit of an image decoding device performing an IBC-based video / image decoding method may include a prediction mode determination unit, a motion vector derivation unit, and a prediction sample derivation unit. The prediction unit of the image decoding device can determine a prediction mode for the current block in the prediction mode determination unit based on received prediction mode information, derive the motion vector of the current block in the motion vector derivation unit based on received motion vector information, and derive a prediction sample of the current block in the prediction sample derivation unit.

[0192] The image decoding device can generate a residual sample for the current block based on the received residual information (S1640). The image decoding device can generate a reconstructed sample for the current block based on the predicted sample and the residual sample, and generate a reconstructed picture based on this (S1650). As previously mentioned, in-loop filtering procedures and the like can then be further applied to the reconstructed picture.

[0193] As mentioned above, a single unit (for example, a coding unit CU) can contain a luma block (luma CB (coding block)) and a chroma block (chroma CB). In this case, the luma block and its corresponding chroma block may have the same motion information (e.g., motion vectors) or they may have different motion information. For example, the motion information of a chroma block can be derived based on the motion information of a luma block, so that the luma block and its corresponding chroma block can have the same motion information.

[0194] Overview of Chroma Format

[0195] The following describes chroma formats. Images can be encoded with encoded data that includes a luma component (e.g., Y) array and two chroma component (e.g., Cb, Cr) arrays. For example, one pixel in an encoded image may contain a luma sample and a chroma sample. A chroma format can be used to indicate the configuration format of luma and chroma samples, and a chroma format can also be called a color format.

[0196] In one embodiment, the image can be encoded in various chroma formats such as monochrome, 4:2:0, 4:2:2, and 4:4:4. In monochrome sampling, there may be one sample array, which may be a luma array. In 4:2:0 sampling, there may be one luma sample array and two chroma sample arrays, each of which may have half the height and half the width of the luma array. In 4:2:2 sampling, there may be one luma sample array and two chroma sample arrays, each of which may have the same height as the luma array and half the width of the luma array. In 4:4:4 sampling, there may be one luma sample array and two chroma sample arrays, each of which may have the same height and width as the luma array.

[0197] For example, in 4:2:0 sampling, the chroma sample can be located at the lower end of the corresponding luma sample. In 4:2:2 sampling, the chroma sample can be superimposed on the position of the corresponding luma sample. In 4:4:4 sampling, both the luma and chroma samples can be located in superimposed positions.

[0198] The chroma format used in the encoding and decoding devices may be predetermined. Alternatively, the chroma format may be signaled from the encoding device to the decoding device for adaptive use in the encoding and decoding devices. In one embodiment, the chroma format may be signaled based on at least one of chroma_format_idc and separate_colour_plane_flag. At least one of chroma_format_idc and separate_colour_plane_flag may be signaled via a higher-level syntax such as DPS, VPS, SPS, or PPS. For example, chroma_format_idc and separate_colour_plane_flag may be included in the SPS syntax as shown in Figure 21.

[0199] On the other hand, Figure 22 shows an example of chroma format classification using signaling of chroma_format_idc and separate_colour_plane_flag. chroma_format_idc may be information indicating the chroma format applied to the encoded image. separate_colour_plane_flag can indicate whether the color array is processed separately in a particular chroma format. For example, the first value of chroma_format_idc (e.g., 0) may indicate monochrome sampling. The second value of chroma_format_idc (e.g., 1) may indicate 4:2:0 sampling. The third value of chroma_format_idc (e.g., 2) may indicate 4:2:2 sampling. The fourth value of chroma_format_idc (e.g., 3) may indicate 4:4:4 sampling.

[0200] In 4:4:4 sampling, the following can be applied based on the value of separate_colour_plane_flag. If the value of separate_colour_plane_flag is the first value (e.g., 0), each of the two chroma arrays can have the same height and width as the luma array. In this case, the value of ChromaArrayType, which indicates the type of chroma sample array, can be set to be the same as chroma_format_idc. If the value of separate_colour_plane_flag is the second value (e.g., 1), the luma, Cb, and Cr sample arrays can be processed separately, so that each can be processed like a monochrome sampled picture. In this case, ChromaArrayType can be set to 0.

[0201] Intra prediction for chromablock

[0202] When intraprediction is performed on the current block, predictions can be made for the luma component block (luma block) and the chroma component block (chroma block) of the current block. In this case, the intraprediction mode for the chroma block can be set separately from the intraprediction mode for the luma block.

[0203] For example, the intra-prediction mode for a chroma block can be indicated based on intra-chroma prediction mode information, which can be signaled in the form of an intra_chroma_pred_mode syntax element. As an example, the intra-chroma prediction mode information can refer to one of the following: Planar mode, DC mode, vertical mode, horizontal mode, DM (Derived Mode), or CCLM (Cross-component linear model) mode. Here, Planar mode can refer to intra-prediction mode 0, DC mode to intra-prediction mode 1, vertical mode to intra-prediction mode 26, and horizontal mode to intra-prediction mode 10. DM can also be called direct mode. CCLM can also be called LM (linear model).

[0204] On the other hand, DM and CCLM are dependent intra-prediction modes that predict the chroma block using information from the luma block. DM can indicate a mode where the same intra-prediction mode used for the luma component is applied as the intra-prediction mode for the chroma component. CCLM can indicate an intra-prediction mode where, in the process of generating a prediction block for the chroma block, the reconstructed sample of the luma block is subsampled, and then the sample derived by applying CCLM parameters α and β to the subsampled sample is used as the prediction sample for the chroma block.

[0205]

number

[0206] Here, pred c (i,j) can represent a predicted sample of the (i,j) coordinates of the current chroma block within the current CU. L'(i,j) can represent a reconstructed sample of the (i,j) coordinates of the current luma block within the CU. For example, the rec L '(i,j) can represent the down-sampled reconstructed sample of the current Lumablock. The linear model coefficients α and β can be signaled, or they can be derived from the surrounding sample.

[0207] Virtual Pipeline Data Unit

[0208] For pipeline processing within a picture, virtual pipeline data units (VPDUs) can be defined. These VPDUs can be defined as non-overlapping units within a single picture. In a hardware decoder, successive VPDUs can be processed simultaneously by multiple pipeline stages. The VPDU size in most pipeline stages can be roughly proportional to the buffer size. Therefore, keeping the VPDU size small is important from a hardware perspective when considering the buffer size. In most hardware decoders, the VPDU size can be set to be the same as the maximum transform block (TB) size. For example, the VPDU size may be 64x64 (64x64 luma samples). Furthermore, the VPDU size can be modified (increased or decreased) to account for the ternary (TT) and / or binary (BT) partitions described above in the VVC.

[0209] On the other hand, in order to maintain the VPDU size at a 64x64 luma sample size, the division of CUs as shown in Figure 23 can be restricted. More specifically, at least one of the following restrictions can be applied.

[0210] Restriction 1: Ternary tree partitioning (TT) is not permitted for CUs that are 128 in width or height, or that are 128 in both width and height.

[0211] Restriction 2: Horizontal binary tree partitioning (BT) is not allowed for CUs of size 128 × N (where N is an integer less than or equal to 64 and greater than 0) (for example, horizontal binary tree partitioning is not allowed for CUs with a width of 128 and a height less than 128).

[0212] Restriction 3: Vertical binary tree partitioning (BT) is not allowed for CUs of size N × 128 (where N is an integer less than or equal to 64 and greater than 0) (for example, vertical binary tree partitioning is not allowed for CUs with a height of 128 and a width less than 128).

[0213] Problem with the maximum size limit of chroma blocks for pipeline processing

[0214] As described above regarding the partitioning structure and transformation process, a CU can be divided to generate multiple TUs. If the size of a CU is larger than the maximum TU size, the CU can be divided into multiple TUs. This allows transformations and / or inverse transformations to be performed on each TU. Generally, the maximum TU size for a Luma block can be set to the maximum available transformation size that the encoding and / or decoding device can perform. An example of CU and TU division according to one embodiment is shown in Figures 24 to 26.

[0215] Figure 24 shows an example of a TU generated by dividing the luma CU and chroma CU according to one embodiment. In one embodiment, the maximum size of the luma CU can be 64 × 64, the maximum available conversion size can be 32 × 32, and non-square TUs can not be allowed. Thus, the maximum size of the luma component conversion block can be 32 × 32. In such an embodiment, the maximum TU size can be set by the following formula.

[0216] [Math 3]

[0217] maxTbSize=(cIdx==0)?MaxTbSizeY:MaxTbSizeY / max(SubWidthC,SubHeightC)

[0218] In the above formula, maxTbSize is the maximum size of the transformation block (TB), and cIdx may be the color component of that block. cIdx0 can represent the luma component, 1 the Cb chroma component, and 2 the Cr chroma component. MaxTbSizeY is the maximum size of the luma component transformation block, SubWidthC is the ratio of the width of the chroma block to the width of the luma block, SubHeightC is the ratio of the height of the chroma block to the height of the luma block, and max(A,B) is a function that returns the larger of A and B as the result value.

[0219] According to the above formula, in the embodiment, in the case of a luma block, the maximum size of the conversion block can be set to the maximum size of the luma component conversion block. Here, the maximum size of the luma component conversion block is a value set during encoding and can be signaled from the encoding device to the decoding device via the bitstream.

[0220] Furthermore, in the above embodiment, the maximum size of the chroma block conversion block can be set to the value obtained by dividing the maximum size of the luma component conversion block by the larger of SubWidthC and SubHeightC. Here, SubWidthC and SubHeightC can be determined based on chroma_format_idc and separate_colour_plane_flag, which are signaled from the encoding device to the decoding device via the bitstream, as shown in Figure 23.

[0221] According to the above formula, in the embodiment, the maximum size of the conversion block can be determined to be one of the minimum width and minimum height that the conversion block can have. As a result, the TU division of the luma block and chroma block in the embodiment can be performed as shown in Figure 24. For example, as shown in Figure 24, in the case of a chroma block having a 4:2:2 format, by determining the maximum size of the conversion block to 16, the chroma CU can be divided into a large number of conversion blocks in a manner different from the division of the luma CU into conversion blocks.

[0222] Figure 25 shows an example of a TU generated by dividing the luma CU and chroma CU according to another embodiment. In one embodiment, the maximum size of the luma CU can be 128 × 128, the maximum available conversion size can be 64 × 64, and non-square TUs may not be allowed. Thus, the maximum size of the luma component conversion block can be 64 × 64. In such an embodiment, the maximum size of the conversion block can be set as shown in the following formula. In the following formula, min(A,B) may be a function that returns the smaller value of A and B.

[0223] [Math 4]

[0224] maxTbSize=(cIdx==0)?MaxTbSizeY:MaxTbSizeY / min(SubWidthC,SubHeightC)

[0225] On the other hand, by applying the larger of the width and height of the block as the maximum size of the transformation block according to the above formula, the luma CU and chroma CU can be divided into multiple TUs, as shown in the example in Figure 25.

[0226] In the examples in Figures 24 and 25, a chroma CU with a 4:2:2 format is divided into TUs in a different manner than the TU division form of the corresponding luma CU. However, when encoding / decoding chroma blocks by referencing luma blocks, as in the DM mode and CCLM mode for predicting chroma blocks mentioned above, it is efficient to process the encoding (or decoding) of the corresponding chroma block immediately after encoding (or decoding) the luma block that corresponds to the chroma block, in order to reduce delay in pipeline processing and save memory.

[0227] However, in the example in Figure 24, after encoding one luma conversion block 2411, encoding of two chroma conversion blocks 2421 and 2423 must be performed, which requires separate processing in relation to other color formats (4:4:4 or 4:2:0). Also, in the example in Figure 25, after encoding of two luma conversion blocks 2511 and 2512, encoding of one chroma conversion block 2521 must be performed. Thus, the TU partitioning method described above has the problem that when using the 4:2:2 format, the luma blocks and their corresponding chroma blocks do not match, so a separate process must be added for pipeline processing, or pipeline processing may not be possible at all.

[0228] Maximum size limit of chroma transformation blocks for pipeline processing

[0229] The following describes how to set the maximum transformation block size for the chromaCU so that the conditions for performing the aforementioned VPDU are met.

[0230] Figure 26 shows an example of a TU generated by partitioning the luma CU and chroma CU according to another embodiment. In this embodiment, the maximum size of the luma CU can be 128 × 128, the maximum available conversion size can be 64 × 64, and the partitioning of non-square TUs can be permitted. This allows the maximum size of the luma component conversion block to be 64 × 64.

[0231] As shown in Figure 26, for the partitioning of a non-square TU, the maximum size of the transformation block can be defined with respect to both width and height. For example, the maximum size of the transformation block can be defined by defining the maximum width (maxTbWidth) and maximum height (maxTbHeight) of the transformation block as shown in the following formulas.

[0232] [Number 5]

[0233] maxTbWidth=(cIdx==0)?MaxTbSizeY:MaxTbSizeY / SubWidthC

[0234] [Number 6]

[0235] maxTbHeight=(cIdx==0)?MaxTbSizeY:MaxTbSizeY / SubHeightC

[0236] As in the embodiment described above, by defining the maximum size of the conversion block by width and height, even in the case of a chroma CU with a 4:2:2 format, as in the example in Figure 26, the chroma CU can be divided into TUs in the same way as the TU division form of the corresponding luma CU. As a result, the TUs of the chroma CU are divided to correspond to the TUs of the luma CU, allowing the encoding (or decoding) of the corresponding chroma block to be processed immediately after the encoding (or decoding) of the luma block, thereby reducing the delay in pipeline processing.

[0237] Maximum size limit of chroma transformation blocks in interpretation mode and IBC prediction mode

[0238] The following describes the execution of the inter-prediction mode and IBC prediction mode with the maximum size limit of the chroma transformation block for chroma pipeline processing described above applied. The encoding and decoding devices can perform inter-prediction and IBC prediction by limiting the maximum size of the chroma transformation block as described below, and their operations can correspond to each other. Furthermore, the following description of inter-prediction can be directly applied to the IBC prediction mode. Accordingly, the inter-prediction operation of the decoding device according to the following embodiment will be described.

[0239] A decoding device according to one embodiment can perform interpretation and generate chroma prediction blocks predSamplesL of size (cbWidth) × (cbHeight) and chroma prediction blocks predSamplesCb and predSamplesCr of size (cbWidth / SubWidthC) × (cbHeight / SubHeightC). Here, cbWidth may be the current width of the CU, and cbHeight may be the current height of the CU.

[0240] The decoding device can then generate chroma residual blocks resSamplesL of size (cbWidth) × (cbHeight) and chroma residual blocks resSamplesCr and resSamplesCb of size (cbWidth / SubWidthC) × (cbHeight / SubHeightC). Finally, the decoding device can generate a reconstructed block using the prediction block and residual blocks.

[0241] The following describes a method by which a decoding device according to one embodiment limits the maximum size of the chroma transform block in order to generate residual blocks of CU encoded in interprediction mode. The decoding device can generate a reconstructed block using the residual blocks generated in this step.

[0242] A decoding device according to one embodiment can obtain the following information directly from the bitstream or derive it from other information obtained from the bitstream in order to generate a residual block of size (nTbW) × (nTbH) of the CU encoded in interprediction mode. Here, nTbW and nTbH may be set to the current width cbWidth and current height cbHeight of the CU.

[0243] - The sample position (xTb0, yTb0) indicates the position of the top-left sample of the current transformation block relative to the position of the top-left sample of the current picture.

[0244] - The parameter nTbW indicates the current width of the conversion block.

[0245] - The parameter nTbH indicates the height of the current transformation block.

[0246] - The parameter cIdx currently indicates the color component of the CU.

[0247] The decoding device can derive the maximum width maxTbWidth and maximum height maxTbHeight of the conversion block from the input information as follows:

[0248] [Number 7]

[0249] maxTbWidth=(cIdx==0)?MaxTbSizeY:MaxTbSizeY / SubWidthC

[0250] [Number 8]

[0251] maxTbHeight=(cIdx==0)?MaxTbSizeY:MaxTbSizeY / SubHeightC

[0252] Furthermore, the decoding device can currently determine the upper left sample position (xTbY, yTbY) of the transformed block based on whether the current CU is a luma component or a chroma component, as follows:

[0253] [Number 9]

[0254] (xTbY,yTbY)=(cIdx==0)?(xTb0,yTb0):(xTb0*SubWidthC,yTb0*SubHeightC)

[0255] As shown in the above formula, if the current transformation block is a chroma block, the maximum width and height of the transformation block and the upper-left sample position of the current transformation block can be determined based on the chroma format to reflect the size of the chroma block determined by the chroma format of the current transformation block.

[0256] The decoding device can generate residual blocks by performing the following procedure, which will be explained with reference to Figure 27. First, the decoding device can determine whether or not to divide the current transformed block (S2710). For example, the decoding device can determine whether or not to divide the current transformed block based on whether the width and height of the current transformed block are greater than the width and height of the maximum transformed block. For example, if nTbW is greater than maxTbWidth or nTbH is greater than maxTbHeight, the decoding device can decide to divide the current transformed block and generate lower-level transformed blocks.

[0257] Currently, when dividing a conversion block into lower conversion blocks, as described above, the decoding device can derive the width newTbW and height newTbH of the lower conversion block as shown in the following formula (S2720).

[0258] [Number 10]

[0259] newTbW=(nTbW>maxTbWidth)?(nTbW / 2):nTbW

[0260] [Number 11]

[0261] newTbH=(nTbH>maxTbHeight)?(nTbH / 2):nTbH

[0262] Next, the decoding device can generate residual blocks using lower-conversion blocks that divide the current conversion block (S2730). In one embodiment, as shown in Figure 26, the current conversion block is a conversion block having the width and height of a chroma CU in a 4:2:2 format, and the lower-conversion blocks may be first lower-conversion blocks 2621 to fourth lower-conversion blocks 2624 that divide it into non-square fours.

[0263] First, the decoding device can generate a residual block for the first downconversion block. Referring to Figure 26, the first downconversion block 2621 can be identified by the sample position (xTb0, yTb0), the width of the downconversion block newTbW, and the height of the downconversion block newTbH. The decoding device can now generate the residual block of the first downconversion block 2621 using the color component cIdx of the current CU. Based on this, the decoding device can generate a modified reconstructed picture. Subsequently, in-loop filtering can be performed on the reconstructed picture.

[0264] Next, if nTbW is greater than maxTbWidth, the decoder can generate a residual block for the second downconversion block. The second downconversion block 2622 can be specified by the sample position (xTb0 + newTbW, yTb0), the width of the downconversion block newTbW, and the height of the downconversion block newTbH. The decoder can now generate the residual block for the second downconversion block 2622 using the color component cIdx of the CU. Based on this, the decoder can generate a modified reconstructed picture. Then, in-loop filtering can be performed on the reconstructed picture.

[0265] Next, when nTbH is greater than maxTbHeight, the decoding device can generate a residual block for the third lowest transformation block. The third lowest transformation block 2623 can be specified by the sample position (xTb0, yTb0 + newTbH), the width newTbW of the lowest transformation block, and the height newTbH of the lowest transformation block. Similarly to the above, the decoding device can generate a residual block using the color component cIdx of the current CU.

[0266] Next, when nTbW is greater than maxTbWidth and nTbH is greater than maxTbHeight, the decoding device can generate a residual block for the fourth lowest transformation block. The fourth lowest transformation block 2624 can be specified by the sample position (xTb0 + newTbW, yTb0 + newTbH), the width newTbW of the lowest transformation block, and the height newTbH of the lowest transformation block. Similarly to the above, the decoding device can generate a residual block using the color component cIdx of the current CU.

[0267] On the other hand, when the current transformation block is not divided, the decoding device can perform inter prediction as follows. For example, when nTbW is less than maxTbWidth and nTbH is less than maxTbHeight, the current transformation block cannot be divided. In such a case, the decoding device can generate a residual block for the inter prediction mode by performing a scaling and transformation process with the sample position (xTbY, xTbY), the color component cIdx of the current CU, the transformation block width nTbW, and the transformation block height nTbH as inputs. The decoding device can generate a modified reconstructed picture based on this. Subsequently, in-loop filtering can be performed on the reconstructed picture.

[0268] Maximum size limit of chroma transformation block in intra predictive mode

[0269] The following describes the execution of the intra-prediction mode with the maximum size limit of the chroma transformation block for chroma pipeline processing described above applied. The encoder and decoder can perform intra-prediction with the maximum size limit of the chroma transformation block limited as described below, and their operations correspond to each other. The operation of the decoder will be described below.

[0270] A decoding device according to one embodiment can generate a reconstructed picture by performing intra-prediction. In-loop filtering can be performed on the reconstructed picture. To perform intra-prediction, the decoding device according to one embodiment can obtain the following information directly from the bitstream or derive it from other information obtained from the bitstream.

[0271] - The sample position (xTb0, yTb0) indicates the position of the top-left sample of the current transformation block relative to the position of the top-left sample of the current picture.

[0272] - The parameter nTbW indicates the current width of the conversion block.

[0273] - The parameter nTbH indicates the height of the current transformation block.

[0274] -The parameter predModeIntra indicates the current CU intra-prediction mode.

[0275] - The parameter cIdx currently indicates the color component of the CU.

[0276] The decoding device can derive the maximum width maxTbWidth and maximum height maxTbHeight of the conversion block from the input information as follows:

[0277] [Number 12]

[0278] maxTbWidth=(cIdx==0)?MaxTbSizeY:MaxTbSizeY / SubWidthC

[0279] [Number 13]

[0280] maxTbHeight=(cIdx==0)?MaxTbSizeY:MaxTbSizeY / SubHeightC

[0281] Furthermore, the decoding device can currently determine the upper left sample position (xTbY, yTbY) of the transformed block based on whether the current CU is a luma component or a chroma component, as follows:

[0282] [Number 14]

[0283] (xTbY,yTbY)=(cIdx==0)?(xTb0,yTb0):(xTb0*SubWidthC,yTb0*SubHeightC)

[0284] Hereinafter, the decoding device can perform intra prediction by performing the following procedure. This will be described with reference to FIG. 28. First, the decoding device can determine whether to divide the current transform block (S2810). For example, the decoding device can determine whether to divide the current transform block based on whether the width and height of the current transform block are greater than the width and height of the maximum transform block. In addition to this, the decoding device can also determine whether to divide by further considering whether ISP (Intra Sub-partition) is applied to the current CU. For example, when nTbW is greater than maxTbWidth or nTbH is greater than maxTbHeight, the decoding device can determine to divide the current transform block and perform intra prediction. Also, the decoding device can determine to divide the current transform block and perform intra prediction only when ISP is not applied to the current CU in such a case (for example, the value of IntraSubpartitionSplitType is NO_ISP_SPLIT. That is, ISP is not applied to the current CU).

[0285] When dividing the current transform block into sub-transform blocks, the decoding device can derive the width newTbW and height newTbH of the sub-transform block as follows (S2820).

[0286] [Equation 15]

[0287] newTbW=(nTbW>maxTbWidth)?(nTbW / 2):nTbW

[0288] [Equation 16]

[0289] newTbH=(nTbH>maxTbHeight)?(nTbH / 2):nTbH

[0290] This will be explained with reference to Figure 26. In one embodiment, the width nTbW of the current conversion block may be the width of the chroma CU, and the height nTbH of the current conversion block may be the height of the chroma CU. In such an embodiment, the width newTbW and height newTbH of the lower conversion block may be determined by the width and height of the conversion block 2621 that divides the chroma CU. That is, in such an embodiment, the current conversion block is a conversion block having the width and height of a chroma CU in a 4:2:2 format, and the lower conversion blocks may be the first lower conversion block 2621 to the fourth lower conversion block 2624 that divide it into non-square fours.

[0291] Next, the decoding device can perform intraprediction using the lower-conversion blocks that divide the current conversion block (S2830). First, the decoding device can perform intraprediction on the first lower-conversion block. Referring to Figure 26, the first lower-conversion block 2621 can be identified by the sample position (xTb0, yTb0), the width of the lower-conversion block newTbW, and the height of the lower-conversion block newTbH. The decoding device can perform intraprediction of the first lower-conversion block 2621 using the current CU's intraprediction mode predModeIntra and the current CU's color component cIdx. This allows a modified reconstructed picture to be generated for the first lower-conversion block 2621.

[0292] For example, the decoding device can generate a predicted sample matrix (predSamples) of size (newTbW) × (newTbH) by performing an intra-sample prediction process. For example, the decoding device can perform the intra-sample prediction process using the sample position (xTb0, yTb0), the intra-prediction mode (predModeIntra), the transformation block width (nTbW)newTbW, the transformation block height (nTbH)newTbH, the coding block width (nCbW)nTbW, the coding block height (nCbH)nTbH, and the value of the parameter cIdx.

[0293] Furthermore, the decoding device can perform scaling and transformation processes to generate a residual sample matrix resSamples of size (newTbW) × (newTbH). For example, the decoding device can perform scaling and transformation processes based on the sample position (xTb0, yTb0), the value of the parameter cIdx, the transformation block width (nTbW)newTbW, and the transformation block height (nTbH)newTbH.

[0294] Furthermore, the decoding device can generate a restored picture by performing a picture restoration process on the color components. For example, the decoding device can perform a picture restoration process on the color components by setting the transformation block position to (xTb0, yTb0), the transformation block width (nTbW) to newTbW, the transformation block height (nTbH) to newTbH, and using the value of the parameter cIdx, and using a prediction sample matrix predSamples of size (newTbW) × (newTbH) and a residual sample matrix resSamples of size (newTbW) × (newTbH).

[0295] Next, if nTbW is greater than maxTbWidth, the decoder can perform an intra-prediction for the second backtransform block. The second backtransform block 2622 can be identified by the sample position (xTb0 + newTbW, yTb0), the width of the backtransform block newTbW, and the height of the backtransform block newTbH. The decoder can perform an intra-prediction for the second backtransform block 2622 using the current CU's intra-prediction mode predModeIntra and the current CU's color component cIdx. The intra-prediction for the second backtransform block 2622 can be performed for its sample position in the same way as the intra-prediction for the first backtransform block 2621. This allows a modified reconstructed picture to be generated for the second backtransform block 2622.

[0296] Next, if nTbH is greater than maxTbHeight, the decoder can perform intraprediction on the third lower-transform block. The third lower-transform block 2623 can be identified by the sample position (xTb0, yTb0 + newTbH), the width of the lower-transform block newTbW, and the height of the lower-transform block newTbH. As before, the decoder can perform intraprediction using the current CU's intraprediction mode predModeIntra and the current CU's color component cIdx.

[0297] Next, if nTbW is greater than maxTbWidth and nTbH is greater than maxTbHeight, the decoder can perform intraprediction for the fourth lower-transform block. The fourth lower-transform block 2624 can be identified by the sample position (xTb0 + newTbW, yTb0 + newTbH), the width of the lower-transform block newTbW, and the height of the lower-transform block newTbH. As described above, the decoder can perform intraprediction using the current CU's intraprediction mode predModeIntra and the current CU's color component cIdx.

[0298] On the other hand, if the decoder does not currently split the transformed block, it can perform an intra prediction as follows: For example, if nTbW is less than maxTbWidth and nTbH is less than maxTbHeight, or if ISP is currently applied to the CU (for example, if the value of IntraSubpartitionSplitType is not NO_ISP_SPLIT), the transformed block may not currently be split.

[0299] First, the decoding device can derive the parameters nW, nH, numPartsX, and numPartsY as shown in the following equation.

[0300] [Number 17]

[0301] nW=IntraSubPartitionsSplitType==ISP_VER_SPLIT?nTbW / NumIntraSubPartitions:nTbW

[0302] nH=IntraSubPartitionsSplitType==ISP_HOR_SPLIT?nTbH / NumIntraSubPartitions:nTbH

[0303] numPartsX=IntraSubPartitionsSplitType==ISP_VER_SPLIT?NumIntraSubPartitions:1

[0304] numPartsY=IntraSubPartitionsSplitType==ISP_HOR_SPLIT?NumIntraSubPartitions:1

[0305] In the above formula, IntraSubPartitionsSplitType indicates the current ISP partition type of the CU, ISP_VER_SPLIT indicates a vertical ISP partition, and ISP_HOR_SPLIT indicates a horizontal ISP partition. NumIntraSubPartitions indicates the number of ISP subpartitions.

[0306] Next, the decoding device can generate a predicted sample matrix of size (nTbW) × (nTbH) called predSamples by performing an intra-sample prediction process. For example, the decoding device can perform the intra-sample prediction process using the sample position (xTb0 + nW * xPartIdx, yTb0 + nH * yPartIdx), the intra-prediction mode predModeIntra, the width of the transformation block (nTbW)nW, the height of the transformation block (nTbH)nH, the width of the coding block (nCbW)nTbW, the height of the coding block (nCbH)nTbH, and the value of the parameter cIdx. Here, the value of the partition index xPartIdx can range from 0 to numPartX-1, and the value of yPartIdx can range from 0 to numPartsY-1.

[0307] Next, the decoding device can generate a residual sample matrix of size (nTbW) × (nTbH) resSamples by performing scaling and transformation processes. For example, the decoding device can perform scaling and transformation processes based on the sample position (xTbY + nW * xPartIdx, yTbY + nH * yPartIdx), the value of the parameter cIdx, the width of the transformation block (nTbW)nW, and the height of the transformation block (nTbH)nH.

[0308] Next, the decoding device can generate a restored picture by performing a picture restoration process on the color components. For example, the decoding device can perform a picture restoration process on the color components by setting the transformation block position to (xTb0+nW*xPartIdx, yTb0+nH*yPartIdx), setting the width of the transformation block (nTbW) to nW, setting the height of the transformation block (nTbH) to nH, and using a pre-set value of cIdx, and using a prediction sample matrix of size (nTbW)×(nTbH) predSamples and a residual sample matrix of size (nTbW)×(nTbH) resSamples.

[0309] Encoding method

[0310] The following describes how an encoding device according to one embodiment performs encoding using the method described above, with reference to Figure 29. The encoding device according to one embodiment includes a memory and at least one processor, and the at least one processor can perform the following encoding method.

[0311] First, the encoding device can divide the image and determine the current block (S2910). Next, the encoding device can generate an interprediction block of the current block (S2920). Next, the encoding device can generate a residual block of the current block based on the interprediction block (S2930). Next, the encoding device can encode the interprediction mode information of the current block (S2940). At this time, the residual block is encoded based on the size of the transformation block of the current block, and the size of the transformation block can be determined based on the color components of the current block.

[0312] More specifically, the position of the upper-left sample of the conversion block can be determined based on the position and color format of the upper-left sample of the Luma block corresponding to the current block.

[0313] Furthermore, if the conversion block is divided into multiple sub-conversion blocks, the upper left position of the sub-conversion block can be determined based on the maximum width and maximum height of the conversion block. For example, the maximum width of the conversion block can be determined based on the maximum size and color format of the conversion block of the Luma block corresponding to the current block, and the maximum height of the conversion block can be determined based on the maximum size and color format of the conversion block of the Luma block corresponding to the current block.

[0314] Furthermore, if the current block is a chroma block and the width of the transformation block is greater than the maximum width of the transformation block, the current block can be vertically divided to generate a plurality of lower transformation blocks. The plurality of lower transformation blocks include a first lower transformation block and a second lower transformation block, the width of the first lower transformation block is determined to be the maximum width of the transformation block, and the upper-left coordinate of the second lower transformation block can be determined to be a value that is separated to the right from the upper-left coordinate of the first transformation block by the maximum width of the transformation block.

[0315] Furthermore, if the current block is a chroma block and the height of the transformation block is greater than the maximum height of the transformation block, the current block can be horizontally divided to generate a plurality of lower transformation blocks. The plurality of lower transformation blocks include a third lower transformation block and a fourth lower transformation block, the height of the third lower transformation block is determined to be the maximum height of the transformation block, and the upper left coordinate of the fourth lower transformation block can be determined to be a value that is separated downward by the maximum height of the transformation block from the upper left coordinate of the first lower transformation block.

[0316] If the color component of the current block is a chroma component, the size of the conversion block can be determined based on the color format. More specifically, the width of the conversion block can be determined based on the maximum width of the conversion block, and the maximum width of the conversion block can be determined based on the maximum size and color format of the conversion block of the chroma block corresponding to the current block.

[0317] Furthermore, if the color component of the current block is a chroma component, the height of the conversion block is determined based on the maximum height of the conversion block, and the maximum height of the conversion block can be determined based on the maximum size and color format of the conversion block of the chroma block corresponding to the current block.

[0318] For example, if the current block's color format is one in which the width of the chroma block is half the width of the corresponding luma block, and the maximum size of the luma block's conversion block is 64x64, then the maximum size of the chroma block's conversion block can be determined to be 32x64.

[0319] Decryption method

[0320] The following describes how a decoding device according to one embodiment performs decoding using the method described above, with reference to Figure 30. The decoding device according to one embodiment includes a memory and at least one processor, and the at least one processor can perform the following decoding method.

[0321] First, the decoding device can determine the prediction mode of the current block (S3010). Next, if the prediction mode of the current block is an inter-prediction mode, the decoding device can generate a prediction block for the current block based on the inter-prediction mode information (S3020). Next, the decoding device can generate a residual block of the transformed block of the current block based on the transformed block of the current block (S3030). Next, the decoding device can restore the current block based on the prediction block and the residual block of the current block (S3040). At this time, the size of the transformed block can be determined based on the color components of the current block.

[0322] The position of the upper-left sample of the conversion block can be determined based on the position and color format of the upper-left sample of the Luma block corresponding to the current block. Furthermore, if the conversion block is divided into multiple sub-conversion blocks, the upper-left position of the sub-conversion block can be determined based on the maximum width and maximum height of the conversion block. For example, the maximum width of the conversion block can be determined based on the maximum size and color format of the conversion block of the Luma block corresponding to the current block, and the maximum height of the conversion block can be determined based on the maximum size and color format of the conversion block of the Luma block corresponding to the current block.

[0323] Furthermore, if the current block is a chroma block and the width of the transformation block is greater than the maximum width of the transformation block, the current block can be vertically divided to generate a plurality of lower transformation blocks. The plurality of lower transformation blocks include a first lower transformation block and a second lower transformation block, the width of the first lower transformation block is determined to be the maximum width of the transformation block, and the upper-left coordinate of the second lower transformation block can be determined to be a value that is separated to the right from the upper-left coordinate of the first transformation block by the maximum width of the transformation block.

[0324] Furthermore, if the block is a chroma block and the height of the transformation block is greater than the maximum height of the transformation block, the current block can be horizontally divided to generate a plurality of lower transformation blocks. The plurality of lower transformation blocks include a third lower transformation block and a fourth lower transformation block, the height of the third lower transformation block is determined to be the maximum height of the transformation block, and the upper left coordinate of the fourth lower transformation block can be determined to be a value that is separated downward by the maximum height of the transformation block from the upper left coordinate of the first lower transformation block.

[0325] If the color component of the current block is a chroma component, the size of the conversion block can be determined based on the color format. More specifically, the width of the conversion block can be determined based on the maximum width of the conversion block, and the maximum width of the conversion block can be determined based on the maximum size and color format of the conversion block of the chroma block corresponding to the current block.

[0326] Furthermore, if the color component of the current block is a chroma component, the height of the conversion block is determined based on the maximum height of the conversion block, and the maximum height of the conversion block can be determined based on the maximum size and color format of the conversion block of the chroma block corresponding to the current block.

[0327] For example, if the current block's color format is one in which the width of the chroma block is half the width of the corresponding luma block, and the maximum size of the luma block's conversion block is 64x64, then the maximum size of the chroma block's conversion block can be determined to be 32x64.

[0328] Application Examples

[0329] The exemplary methods in this disclosure are presented as a series of actions for clarity of explanation, but this is not intended to restrict the order in which the steps are performed, and each step may be performed simultaneously or in a different order, if necessary. To implement the methods according to this disclosure, the exemplary steps may be further varied, including the remaining steps with some exceptions, or including additional steps with some exceptions.

[0330] In this disclosure, an image encoding device or image decoding device that performs a predetermined operation (step) may perform an operation (step) to confirm the conditions or status of the execution of said operation (step). For example, if it is stated that a predetermined operation is performed when a predetermined condition is satisfied, the image encoding device or image decoding device may perform an operation to confirm whether or not the predetermined condition is satisfied, and then perform the predetermined operation.

[0331] The various embodiments of this disclosure are not intended to list all possible combinations, but rather to illustrate representative aspects of this disclosure. The matters described in the various embodiments may be applied independently or in combination of two or more.

[0332] Furthermore, various embodiments of this disclosure can be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, it can be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0333] Furthermore, the image decoding and image encoding devices to which the embodiments of this disclosure are applied can be included in multimedia broadcasting transceivers, mobile communication terminals, home cinema video equipment, digital cinema video equipment, surveillance cameras, video conferencing equipment, real-time communication equipment such as video communications, mobile streaming equipment, storage media, camcorders, video-on-demand (VoD) service providers, over-the-top (OTT) video equipment, internet streaming service providers, 3D video equipment, image-phone video equipment, and medical video equipment, and can be used to process video signals or data signals. For example, over-the-top (OTT) video equipment can include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, and digital video recorders (DVRs).

[0334] Figure 31 illustrates a content streaming system to which the embodiments of this disclosure can be applied.

[0335] As shown in Figure 31, a content streaming system to which an embodiment of the present disclosure is applied may broadly include an encoding server, a streaming server, a web server, media storage, user equipment, and multimedia input devices.

[0336] The encoding server is responsible for compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and transmitting this bitstream to the streaming server. In other cases, if a multimedia input device such as a smartphone, camera, or video camera directly generates the bitstream, the encoding server can be omitted.

[0337] The bitstream can be generated by an image encoding method and / or image encoding apparatus to which an embodiment of the present disclosure is applied, and the streaming server can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0338] The streaming server transmits multimedia data to the user's device based on the user's request via a web server, and the web server can act as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transmits this to the streaming server, and the streaming server can transmit multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server can play a role in controlling the commands and responses between the devices within the content streaming system.

[0339] The streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from the encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0340] Examples of user devices include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices such as smartwatches, smart glasses, HMDs (head-mounted displays), digital TVs, desktop computers, and digital signage.

[0341] Each server within the aforementioned content streaming system can be operated as a distributed server, in which case the data received from each server can be processed in a distributed manner.

[0342] The scope of this disclosure includes software or machine-executable commands (e.g., operating systems, applications, firmware, programs, etc.) that enable the operation of various embodiments to be performed on a device or computer, and non-transitory computer-readable medium on which such software or commands etc. are stored and can be executed on a device or computer. [Industrial applicability]

[0343] The embodiments described herein can be used for encoding / decoding images.

Claims

1. An image decoding method performed by an image decoding device, The current step is to determine the prediction mode of the block, Based on the fact that the prediction mode of the current block is an inter-prediction mode, the step of generating a prediction block of the current block based on the inter-prediction mode information, A step of determining whether to divide the current block into multiple sub-conversion blocks based on a comparison of the size of the conversion block and the maximum size of the conversion block, The steps include generating a residual block of the current block based on the conversion block, The step of restoring the current block based on the predicted block and the residual block of the current block includes, The maximum size of the conversion block includes the maximum width of the conversion block and the maximum height of the conversion block. The maximum width and maximum height of the transformation block are determined separately based on the color components of the current block. An image decoding method in which, based on the fact that the conversion block is divided into a plurality of lower conversion blocks, the upper left position of one of the lower conversion blocks is determined based on the size of the conversion block.

2. An image encoding method performed by an image encoding device, The steps include determining the current block by dividing the image, The steps include generating an interpretation block for the current block, The steps include generating a residual block of the current block based on the inter prediction block, The step includes encoding interprediction mode information of the current block, The residual block is encoded based on the size of the transformation block of the current block. Whether to divide the conversion block for the current block into multiple sub-conversion blocks is determined based on a comparison of the size of the conversion block and the maximum size of the conversion block. The maximum size of the conversion block includes the maximum width of the conversion block and the maximum height of the conversion block. The maximum width and maximum height of the transformation block are determined separately based on the color components of the current block. An image encoding method in which, based on the fact that the conversion block is divided into a plurality of lower conversion blocks, the upper left position of one of the lower conversion blocks is determined based on the size of the conversion block.

3. A method for transmitting a bitstream generated by an image encoding method, The aforementioned image encoding method is The steps include determining the current block by dividing the image, The steps include generating an interpretation block for the current block, The steps include generating a residual block of the current block based on the inter prediction block, The step includes encoding interprediction mode information of the current block, The residual block is encoded based on the size of the transformation block of the current block. Whether to divide the conversion block for the current block into multiple sub-conversion blocks is determined based on a comparison of the size of the conversion block and the maximum size of the conversion block. The maximum size of the conversion block includes the maximum width of the conversion block and the maximum height of the conversion block. The maximum width and maximum height of the transformation block are determined separately based on the color components of the current block. A method in which, based on the fact that the conversion block is divided into a plurality of subconversion blocks, the upper left position of one of the subconversion blocks is determined based on the size of the conversion block.

Images